Create xp.network

[DimaBryuhanov]

Grant Application Checklist

• The application template has been copied, renamed ( project_name.md) and updated.
• A BTC or Ethereum (DAI) address for the payment of the milestones is provided inside the application.
• I have read and acknowledged the Terms and Conditions.
• The software delivered for this grant will be released under an open-source license specified in the application.
• The total funding amount of the project is below USD $30k for initial grant applications and $100k for follow-up grants.
• The initial PR contains only one commit (squash if needed before submitting your PR).
• The grant will only be announced once the first milestone has been accepted.

[CLAassistant]

All committers have signed the CLA.

[alxs]

Hi @DimaBryuhanov. Thank you for the application. Could you rename the file to .md so it is properly displayed?

[DimaBryuhanov]

Hi, @alxs, Thank you for your comment. I have renamed the file to .md. Feel free to let me know if any other changes are required.

[alxs]

Thanks, one of us will provide more extensive feedback soon. A couple of comments:

• Grants now need to be denominated in USD and can be paid out in either BTC or DAI. Could you update this?
• Your deliverables need to be much more well-defined and explicit. You may treat this as a contract; the level of detail must be enough to later verify that the software meets the specification. You can find some examples of details we're interested in for different grant categories here. Besides, the deliverables need to be quantifiable, potentially useful to other teams and not already present in the ecosystem. I.e. we won't fund things like deliverable 5 of M4 "Publishing the XP Network VM Hub for public use".
• Note that deliverables 0a-c in the application template need to be included in all milestones.

[DimaBryuhanov]

Hi, @alxs, Thanks you for your comments.

Since our last commit we have:

1. Fixed the currency according to your recommendations,
2. Added the points 0.a-c to every Milestone,
3. Expanded the part with the XP.network Relay Chain protocol,
4. Added PoC examples related to the protocol,
5. Added documentation in Rust with the description of the existing functions,
6. Removed the part where we translated from one Smart Contract language to another.

I hope, the application has become cleaner. Pls., let me know if anything else requires fixing.

Your kind comments are highly welcome.

[Noc2]

Thanks for the update. Is it correct that you want to deploy your own relay chain? Why not initially try to contribute to existing solutions and try to improve these? Your xcmp_pallet-poc is at the moment basically just Acala’s implementation anyway. You could also contribute to the psp process Also are you aware of SPREE? And in general XCMP is currently under construction and things will change, see https://w3f.github.io/parachain-implementers-guide/messaging.html#cross-chain-message-passing and https://research.web3.foundation/en/latest/polkadot/XCMP/index.html

[DimaBryuhanov]

Thanks for your kind advice.

1. We didn’t want to deploy our own Relay Chain unless it became required to fulfill our goal.
2. We are precisely suggesting a contribution to an existing solution, which is XCMP, even though it is still a work in progress.
3. We read about SPREE, but we’ve found that it only exists in documentation. We also learned, that development of SPREE will begin after several months when parachians become available. Therefore, we did not rely on it, since it would be impossible to test our protocol in a non-existent environment.

Advantages of our protocol over the existing XCMP and SPREE, allowing parachains to communicate more efficiently are the following:

1. Our protocol allows to mark and store the current state of every transaction in a blob with a unique TopicID. Even though the future implementations of XCMP will allow to create channels between two parachains, TopicIDs will still be useful when multiple transactions are being executed between two parachains simultaneously. Another XCMP feature - ordered delivery - does not provide the same precision as the TopicId does. The TopicId does not rely on the order of messages. It relies on the unique identifier which is the safest pointer.
2. Once the state of a transaction changes or an error occurs, the requesting parachain gets notified. Some of such messages help keep the end user properly informed about the state of events with his/her transaction, others inform the pallets that the transaction has terminated and the memory must be freed from the blob, which might be expensive to store, especially when it is no longer needed.
3. Another feature uses the fact that a pallet, implementing our protocol, is a part of its parent parachain. Therefore, it has no additional overhead in tracking whether the transaction succeeded or failed. It listens to events related to the transaction and notifies the requesting parachian of the result and provides the data of the outcome when it becomes available without being additionally requested about this information.

Please, kindly advise whether the following sequence of events is acceptable:

1. We can keep working on our protocol over the existing XCMP and once it changes, we will adapt our protocol to the new environment of XCMP.
2. We will migrate our protocol to SPREE, once it becomes available. This means that the blob will be stored inside SPREE’s own memory, it will be impossible for a parachain to fake or alter the storage, etc.

[Noc2]

Thanks for the quick reply. Feel free to integrate this information also into the application itself. And you are right that SPREE is at the moment far from ready. I have a few additional questions/comments:

• What about the scalability of your solution? Doesn’t this create a lot of overhead for the relay chain?
• In general it would be great if you could potentially include the psp process into your deliveries: https://github.com/w3f/PSPs or even already start a discussion around it before the grant, see for example PSP2 - Substrate Uniform Resource Identifier PSPs#11. My main concern here is that ideally this should not result in conflicting standards, but maybe rather be an extension and if it’s a conflicting standard then there should be a clear reason for it.
• To be honest, I’m not sure I completely understand the deliverable of your milestones. For example why is the name of your first milestone “VM Hub pallet Move bytecode to Solidity bytecode (pre-MVP)”. Wouldn’t it make sense to initially integrate this as rust substrate pallet?
• Also what do you plan to use for message serialisation/deserialisation (M1). Couldn’t you use SCALE or Protobuf for this?

[DimaBryuhanov]

@Noc2, Thank you for your kind comments and suggestions.

1. We’ve integrated the information explaining why we’re using XCMP and not SPREE as a transport layer of our protocol in the application body.

2. We certainly thought about scalability as well, this is why, even the complete message only adds 64 additional bits to the original TX binary code. 16 bits for the TopicId, 16 bits for the flags and 32 bits for the length of the TX binary. In order to reduce the overhead:

• When notifying about errors, or that a TX has been submitted for execution, only the first 32 bits are attached to the message (16 bits of the TopicId and 16 bits for flags). They will be used to update the blob, where the state of a TX is stored during the number of seconds when a TX is executed in a target parachain. The TX bytecode is not moved when it is not required.
• Should sometime in the future even such tiny notifications create a noticeable overhead, we will
  a. join them into batches
  b. remove the AKN notification, which is required till SPREE becomes available to confirm that the message has been received with the same TX code.

3. We have included the PSP process to every milestone.

4. We’ve fixed the obsolete Milestone names.

5. We’re currently using serde for serialisation since it is mature (over 45 million downloads), it has good documentation and it is well maintained.

• SCALE depends on the same library. But, since we’re not using the entire functionality of SCALE we thought that attaching it would make our project a bit heavier than necessary.
• Protobuf is less mature (4 million downloads). Its documentation is much more scarce and it also depends on serde.

Please, kindly advise, how we can proceed.

[Noc2]

@DimaBryuhanov Thanks for the updates and opening the psp. It seems like others already replied to it and are also uncertain about the scalability of your solution. It probably makes sense to continue the conversation about it there: w3f/PSPs#17.
Xiliang Chen and Jeffrey Burdges, who already looked at your psp, both know more about XCMP than I do. The advantage of initially using SCALE is that it’s the “default” solution for substrate. This makes it easier to initially focus on the protocol itself.

[alxs]

My main concern here is that ideally this should not result in conflicting standards, but maybe rather be an extension and if it’s a conflicting standard then there should be a clear reason for it.

Just to add here that I share this concern and the current PSP doesn't address this, but rather presents XP.network precisely as a conflicting (or complementary) standard. I think the right place to discuss potential improvements to XCM would be the Polkadot (see the number of open issues related to XCM) or XCM Format repositories. Note that you could also work on XCM within the context of a grant, either on any of the areas mentioned in this RFP or on improvements such as what you propose. Feel free to open an issue with your suggestions in one of those repositories if that sounds interesting.

And a couple of other remarks in case you decide to stick to the current application:

• The delivery time isn't really a deliverable. Could you move this outside of the table and instead include testing as indicated in the application template?
• All deliverables in milestone 3 are requirements for every milestone. I suggest you remove this milestone and instead explicitly integrate these into the first two.
• I think there's some confusion with the terminology used under project details. If I understand correctly the "XP.network Relay Chain Protocol" doesn't have anything to do with the relay chain (as I think you realised here), and furthermore I think XP.network wouldn't be "supported by a number of pallets" but provided by a pallet. Correct me if I'm wrong.
• Just to make sure we're talking about the same thing: do I understand it correctly that in its current state XP.network would be, in a nutshell, a reliable messaging protocol that would use XCM as its transport layer?

[DimaBryuhanov]

@alxs, thank you for your swift reply.

You’re right, our protocol adds valuable functionality to the XCMP protocol, which is used as a transport layer for XP.network.

We have implemented the requirements you suggested:

1. Delivery times removed

2. Deliverables from milestone 3 moved to milestones 1 and 2

3. "Relay Chain" removed from the application to make things clearer

4. Our protocol addresses two of the raised issues

• Adding events for UMP message execution results
  Instead of broadcasting events spamming the network, we message only the target pallet. Reducing excessive computation in the blockchain.

• XCM Retry Queue #2702 Suggest several use-cases, where our protocol can come handy:

1. When a nonsense message or erroneous code is sent, a DER (destination error flag) is raised, and if a parachain produces a more wordy explanation, it would be delivered in the TXData section of the blob. Such TX should not be retried in the future.

2. When an unknown token has been transferred to a parachain and the target parachain failed to process the request, an error flag will be raised.
   An END flag will not be raised, therefore the state blob will not be removed.

   This will prevent the assets from being irreversibly lost and will leave the opportunity for the developers to add the token and allow the transaction to be successfully processed at a later time.

3. In case there's not enough gas in the target parachain the process will be the same as above in point 2.

[alxs]

Thanks, the deliverables do look much better now. However I have to say I'm hesitant to support the project in general, as I'd rather see the necessary changes implemented in XCMP than support a complementary protocol. I also have difficulties imagining use cases (of parachains communicating via threads without implementing a dedicated protocol anyway) and find it very hard to justify the $30k price tag.

The PoC is also currently just a fork of Acala's xtokens with 10 new lines of code, plus the message headers implemented as a bit field. I would have liked to see something a bit more convincing. Yours and @VKint's GitHub activity is also almost empty so it's currently hard to verify your experience. Do you have any past projects you've worked on that you could share?

In case there's not enough gas in the target parachain the process will be the same as above in point 2.

Also just wanted to point out that the concept of gas doesn't exist on Polkadot. See https://wiki.polkadot.network/docs/en/learn-transaction-fees

[DimaBryuhanov]

@alxs, thank you for your feedback.

1. XP.network is one of the projects that will require this protocol. We are sure that all Polkadot's projects using SPREE and XCMP will use our protocol as well.

2. Multiple changes have been made on the deliverables. We have adjusted the delivery time and price estimation accordingly.

3. Regarding the POC, please note there are two repositories. There should be 1400+ new lines of code combined.

For example, here you can find more than 1k of new lines:

XP-NETWORK/xcmp_pallet-poc@26f8e49#diff-11c34f40e6bcec5656496389df4175a62fee9ac5809a4b7f369b4492d7e625c2

[alxs]

That is definitely Acala's code. I've attached the diff that shows the extent of the changes made to the xtokens pallet is indeed just about 10 lines. I'm sorry if you weren't aware of this and didn't intend to misrepresent your work, but in either case this is too large a red flag for us to further consider the application. In case you want to apply again, I'd like to ask you to please include a significantly developed PoC with your application and some indication of past experience. Thanks in advance.

git diff

 diff --git a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/Cargo.toml b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/Cargo.toml
index 667ae09..4c04d19 100755
--- a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/Cargo.toml
+++ b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/Cargo.toml
@@ -1,39 +1,41 @@
[package]
-name = "orml-xtokens"
-description = "Cross-chain token transfer."
+name = "orml-xmessage"
+description = "Cross-chain message transfer."
repository = "https://github.com/open-web3-stack/open-runtime-module-library/tree/master/xtokens"
license = "Apache-2.0"
version = "0.4.1-dev"
-authors = ["Acala Developers"]
+authors = ["Acala Developers", "XP-Network"]
edition = "2018"

[dependencies]
serde = { version = "1.0.124", optional = true }
codec = { package = "parity-scale-codec", version = "2.0.0", default-features = false }
-sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3", default-features = false }
-sp-io = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3", default-features = false }
-sp-std = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3", default-features = false }
+sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "statemint", default-features = false }
+sp-io = { git = "https://github.com/paritytech/substrate", branch = "statemint", default-features = false }
+sp-std = { git = "https://github.com/paritytech/substrate", branch = "statemint", default-features = false }

-frame-support = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3", default-features = false }
-frame-system = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3", default-features = false }
+frame-support = { git = "https://github.com/paritytech/substrate", branch = "statemint", default-features = false }
+frame-system = { git = "https://github.com/paritytech/substrate", branch = "statemint", default-features = false }

-cumulus-primitives-core = { git = "https://github.com/paritytech/cumulus", branch = "polkadot-v0.9.3", default-features = false }
+cumulus-primitives-core = { git = "https://github.com/paritytech/cumulus", branch = "statemint", default-features = false }

-xcm = { git = "https://github.com/paritytech/polkadot", branch = "release-v0.9.3", default-features = false }
+xcm = { git = "https://github.com/paritytech/polkadot", branch = "statemint", default-features = false }

orml-xcm-support = { path = "../xcm-support", default-features = false }
orml-traits = { path = "../traits", default-features = false}

+serde_xp_protocol = { git = "https://github.com/xp-network/serde_xp_protocol" }
+
[dev-dependencies]
-sp-core = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3" }
-polkadot-core-primitives = { git = "https://github.com/paritytech/polkadot", branch = "release-v0.9.3" }
-polkadot-parachain = { git = "https://github.com/paritytech/polkadot", branch = "release-v0.9.3" }
-pallet-balances = { git = "https://github.com/paritytech/substrate", branch = "polkadot-v0.9.3" }
+sp-core = { git = "https://github.com/paritytech/substrate", branch = "statemint" }
+polkadot-core-primitives = { git = "https://github.com/paritytech/polkadot", branch = "statemint" }
+polkadot-parachain = { git = "https://github.com/paritytech/polkadot", branch = "statemint" }
+pallet-balances = { git = "https://github.com/paritytech/substrate", branch = "statemint" }
xcm-simulator = { git = "https://github.com/shaunxw/xcm-simulator", branch = "master" }
-cumulus-pallet-xcm-handler = { git = "https://github.com/paritytech/cumulus", branch = "polkadot-v0.9.3" }
-parachain-info = { git = "https://github.com/paritytech/cumulus", branch = "polkadot-v0.9.3" }
-xcm-executor = { git = "https://github.com/paritytech/polkadot", branch = "release-v0.9.3" }
-xcm-builder = { git = "https://github.com/paritytech/polkadot", branch = "release-v0.9.3" }
+cumulus-pallet-xcm-handler = { git = "https://github.com/paritytech/cumulus", branch = "statemint" }
+parachain-info = { git = "https://github.com/paritytech/cumulus", branch = "statemint" }
+xcm-executor = { git = "https://github.com/paritytech/polkadot", branch = "statemint" }
+xcm-builder = { git = "https://github.com/paritytech/polkadot", branch = "statemint" }

orml-tokens = { path = "../tokens", version = "0.4.1-dev" }
orml-traits = { path = "../traits", version = "0.4.1-dev" }
diff --git a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/src/lib.rs b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/src/lib.rs
index c989811..f9970ff 100755
--- a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/src/lib.rs
+++ b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/src/lib.rs
@@ -29,10 +29,12 @@ use sp_runtime::{
	DispatchError,
};
use sp_std::prelude::*;
+use serde_xp_protocol::{XpProtocol, Flags};

use xcm::v0::{
	Junction::*,
	MultiAsset, MultiLocation, Order,
+    AssetInstance,
	Order::*,
	Xcm::{self, *},
};
@@ -54,13 +56,12 @@ pub mod module {
		type Event: From<Event<Self>> + IsType<<Self as frame_system::Config>::Event>;

		/// The balance type.
-		type Balance: Parameter
+		type Message: Parameter
			+ Member
-			+ AtLeast32BitUnsigned
+            + Vec<u8>
			+ Default
			+ Copy
-			+ MaybeSerializeDeserialize
-			+ Into<u128>;
+			+ MaybeSerializeDeserialize;

		/// Currency Id.
		type CurrencyId: Parameter + Member + Clone;
@@ -83,10 +84,8 @@ pub mod module {
	#[pallet::generate_deposit(fn deposit_event)]
	#[pallet::metadata(T::AccountId = "AccountId", T::CurrencyId = "CurrencyId", T::Balance = "Balance")]
	pub enum Event<T: Config> {
-		/// Transferred. \[sender, currency_id, amount, dest\]
-		Transferred(T::AccountId, T::CurrencyId, T::Balance, MultiLocation),
-		/// Transferred `MultiAsset`. \[sender, asset, dest\]
-		TransferredMultiAsset(T::AccountId, MultiAsset, MultiLocation),
+		/// Message Sent \[sender, currency_id, dest\]
+		Message(T::AccountId, T::CurrencyId, MultiLocation),
	}

	#[pallet::error]
@@ -109,48 +108,29 @@ pub mod module {

	#[pallet::call]
	impl<T: Config> Pallet<T> {
-		/// Transfer native currencies.
+		/// XpProtocol Serialization PoC
		#[transactional]
		#[pallet::weight(1000)]
		pub fn transfer(
			origin: OriginFor<T>,
			currency_id: T::CurrencyId,
-			amount: T::Balance,
+			message: Vec<u8>,
			dest: MultiLocation,
		) -> DispatchResultWithPostInfo {
			let who = ensure_signed(origin)?;

-			if amount == Zero::zero() {
-				return Ok(().into());
-			}
-
			let id: MultiLocation = T::CurrencyIdConvert::convert(currency_id.clone())
				.ok_or(Error::<T>::NotCrossChainTransferableCurrency)?;
-			let asset = MultiAsset::ConcreteFungible {
+
+            // topic_id, flags, message
+            let xp_data = XpProtocol::new(2314112141, Flags(0), &message);
+
+			let asset = MultiAsset::ConcreteNonFungible {
				id,
-				amount: amount.into(),
+			    instance: AssetInstance::Blob(serde_xp_protocol::to_bytes(xp_data)),
			};
			Self::do_transfer_multiasset(who.clone(), asset, dest.clone())?;
-			Self::deposit_event(Event::<T>::Transferred(who, currency_id, amount, dest));
-			Ok(().into())
-		}
-
-		/// Transfer `MultiAsset`.
-		#[transactional]
-		#[pallet::weight(1000)]
-		pub fn transfer_multiasset(
-			origin: OriginFor<T>,
-			asset: MultiAsset,
-			dest: MultiLocation,
-		) -> DispatchResultWithPostInfo {
-			let who = ensure_signed(origin)?;
-
-			if Self::is_zero_amount(&asset) {
-				return Ok(().into());
-			}
-
-			Self::do_transfer_multiasset(who.clone(), asset.clone(), dest.clone())?;
-			Self::deposit_event(Event::<T>::TransferredMultiAsset(who, asset, dest));
+			Self::deposit_event(Event::<T>::Message(who, currency_id, dest));
			Ok(().into())
		}
	}
diff --git a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/target/.rustc_info.json b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/target/.rustc_info.json
index 8760a5e..b30064b 100755
--- a/home/aleixo/Software/repos/open-runtime-module-library/xtokens/target/.rustc_info.json
+++ b/home/aleixo/Software/repos/xcmp_pallet-poc/xmessage/target/.rustc_info.json
@@ -1 +1 @@
-{"rustc_fingerprint":5336087126026914120,"outputs":{"17598535894874457435":{"success":true,"status":"","code":0,"stdout":"rustc 1.52.1 (9bc8c42bb 2021-05-09)\nbinary: rustc\ncommit-hash: 9bc8c42bb2f19e745a63f3445f1ac248fb015e53\ncommit-date: 2021-05-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.52.1\nLLVM version: 12.0.0\n","stderr":""},"2797684049618456168":{"success":false,"status":"exit code: 1","code":1,"stdout":"","stderr":"error: `-Csplit-debuginfo` is unstable on this platform\n\n"},"931469667778813386":{"success":true,"status":"","code":0,"stdout":"___\nlib___.rlib\nlib___.so\nlib___.so\nlib___.a\nlib___.so\n/home/aleixo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu\ndebug_assertions\nproc_macro\ntarget_arch=\"x86_64\"\ntarget_endian=\"little\"\ntarget_env=\"gnu\"\ntarget_family=\"unix\"\ntarget_feature=\"fxsr\"\ntarget_feature=\"sse\"\ntarget_feature=\"sse2\"\ntarget_os=\"linux\"\ntarget_pointer_width=\"64\"\ntarget_vendor=\"unknown\"\nunix\n","stderr":""}},"successes":{}}
\ No newline at end of file
+{"rustc_fingerprint":5336087126026914120,"outputs":{"17598535894874457435":{"success":true,"status":"","code":0,"stdout":"rustc 1.52.1 (9bc8c42bb 2021-05-09)\nbinary: rustc\ncommit-hash: 9bc8c42bb2f19e745a63f3445f1ac248fb015e53\ncommit-date: 2021-05-09\nhost: x86_64-unknown-linux-gnu\nrelease: 1.52.1\nLLVM version: 12.0.0\n","stderr":""},"931469667778813386":{"success":true,"status":"","code":0,"stdout":"___\nlib___.rlib\nlib___.so\nlib___.so\nlib___.a\nlib___.so\n/home/aleixo/.rustup/toolchains/stable-x86_64-unknown-linux-gnu\ndebug_assertions\nproc_macro\ntarget_arch=\"x86_64\"\ntarget_endian=\"little\"\ntarget_env=\"gnu\"\ntarget_family=\"unix\"\ntarget_feature=\"fxsr\"\ntarget_feature=\"sse\"\ntarget_feature=\"sse2\"\ntarget_os=\"linux\"\ntarget_pointer_width=\"64\"\ntarget_vendor=\"unknown\"\nunix\n","stderr":""},"2797684049618456168":{"success":false,"status":"exit code: 1","code":1,"stdout":"","stderr":"error: `-Csplit-debuginfo` is unstable on this platform\n\n"}},"successes":{}}
\ No newline at end of file

[DimaBryuhanov]

@alxc, thank you for your time.
We will keep working on the protocol, and will apply with several other projects.