From 5471545d598280e2b01a6ec0a73b7b995cfbbd91 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Fri, 7 May 2021 00:23:25 +0300 Subject: [PATCH 01/20] Create xp.network --- applications/xp.network | 273 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 applications/xp.network diff --git a/applications/xp.network b/applications/xp.network new file mode 100644 index 00000000000..b57641c5c66 --- /dev/null +++ b/applications/xp.network @@ -0,0 +1,273 @@ +# Open Grant Proposal + +- **Project Name:** XP.network VM Hub. +- **Team Name:** XP.network. +- **Payment Address:** BTC: bc1qdpx2e6lejre536ph0csskas888ua3pz5p4pkuj. + +## Project Overview + +### Overview + +XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP.network allows non-developers to build their NFT marketplaces, galleries, museums and many more use-cases without coding. + +In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets. + +Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID". It will be a group of pallets, each acting like a “post office” from a post office network. They will all use our XP Relay Chain protocol. + +Since different blockchains may use different smart contract languages, we are aspiring to create an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which could be further validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has finished or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. + +Because there's no NFT library in Move's standard library we will have to write it as part of this project. This will allow +1. other blockchains to call NFT smart contracts in Move, +2. to run NFT smart contracts in other languages from Move. + +#### Integration + +All the NFT based applications built by our platform will be using the XP Relay Chain protocol we're building. + +The pallets (one for each parahcain / parathread) will act as a “post office network” supporting the protocol that enables nodes to keep track of the “topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. + +### Project Details + +**XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office”. A typical message will include: +```terminal +{ +ID: id, //required to identify that the other blockchain’s reply is related to this request, +CallbackFunction: funcName, // a designated Polkadot Relay Chain callback function, +CallbackArguments: [ ... ], // required for the above function, +To: dest, // indicates the destination parachain / parathread, +Payload: blob // A binary representation of the "intention" +} +``` + +The message inside the binary payload will be structured as follows: + +![img](https://github.com/xp-network/w3f_application/blob/main/XP.network.protocol.png) + +The **XP.network Handshake protocol** will look like this: + +1. An initiating pallet sends a message with a smart contract call to a designated pallet. +2. Once the designated pallet receives the message it unpacks the blob from the payload and returns the confirmation like this: + + a. it swaps the Source & the Destination addresses + + b. it flips the AKN flag from 0 to 1, keeping the other flags in the off state + + c. it checks whether the template of the desired type exists and the argument list matches the requirement. In case at least one does not, the DER (destination error) and END flags are set to 1. + + c. it keeps the rest of the blob intact to prove to the sender that exactly this message was received. + + d. it crafts a Message with the same ID and sends it back for the initiator to confirm that the transaction is being processed or failed. + +4. The initiating pallet checks the integrity of the parcel and whether END or DER flags were not raised and if everything is ok will return the same message (with swapped Sender & Receiver) with the INT(egrity) flag set to 1. Otherwise the INT(egrity) flag will remain 0, but NER(nework error) flag will be raised to indicate that the message was corrupted in the transport layer. +5. Once the message with the Topic_ID arrives to the destination pallet it will check the inegrity flag and if the flag is 1 it will start processing the request. Otherwise, it will retrun the same message, acknowledging broken integrity and proving that the transaction is terminated. +6. Once the request has been processed, submited to the target blockchain and a success / failure result is received form the blockchain, it will craft a new message with the same Topic_ID setting the OK or REJ flags as well as the END flag to 1. +7. Once the above message is received by the initiating pallet it will check its integrity and will pass the result to its blockchain. It will then send the same message back to the counterpart to finish negotiation on the Topic_ID. +8. IER stands for Initiator pallet error. This flag will be raised if there is a technical issue in the initiating pallet. + +Every parachain equipped with our pallet will know how to read such incoming messages. If the message is related to the blockchain this pallet is attached to, it will do the following: + +1. Deserialize the incoming message from bytecode to optcode, +2. Transform the commands and arguments from the optcode to a smart contract in the target language, used by the blockchain it works for. +3. It will pass the generated bytecode to the blockchain for execution. +4. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via 5. Polkadot Relay Chain callbacks with the same id it got the request. +6. The pallet which initiated the transaction will receive the reply and will match the id of the reply with the id of the request. +7. It will deserialize the bytecode of the reply and will transform it to the bytecode of its smart contract language (should it be different from the counterpart’s). + +A pallet implementing this protocol consists of: + +1. **Bytecode Deserializer** - it receives binary input and returns a human readable opcode. +2. **Assembly Code Converter** - it creates the following key - value pairs: + + The chosen smart contract programming language. + + Template number. + + Data to populate the smart contract with. +3. **Bytecode Compiler** - it takes the key - value pairs generated at the previous stage as an input and generates the chosen smart contract language bytecode as its output. +4. **Polkadot parathread** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. + +A set of pre-programmed audited code templates are ready to be populated by the arbitrary data. Once a request is received, the templates are populated with the incoming data and are instantly compiled into transaction ready bytecode. Initially there will be a limited set of ready code templates for each platform. However, new templates will be added on a permanent regular basis. Eventually most possible use cases will be available for each bridged platform. Finally, the original smart contract bytecode will be translated directly into the target language bytecode, removing all limitations. + +#### [PoC](https://github.com/xp-network/xp-compiler) + +Before applying for the grant we have prepared GitHub repositories with the Proof of Concept. + +Since the idea behind the VM Hub is converting one smart contract language bytecode to another we have elaborated the following steps: + +1. We wrote a short smart contract in Solidity. +2. We compiled it to bytecode with the solc compiler. +3. We deserialized the bytecode into opcode. +4. An array of opcodes became a bytecode template: + a. Commands (MSTORE, SSTORE, CALLVALUE, ISZERO, JUMPI, REVERT… ) + b. Data (PUSH 80, PUSH [$] 000000000000000000000000000000000000000001, etc.) +5. We wrote a request in Move which will specify which template we want to use and will provide the data to populate the template with new data. +6. The code in Move is then compiled to bytecode & passed to our Substrate pallet. +7. The pallet deserializes the Move bytecode and extracts the instruction - which template to use and the data to populate it with. +8. The data from the request is used to populate the opcode parameters in the template. +9. The template is reassembled to bytecode. +10. We run the two bytecodes (from steps 2 & 9) in the Ethereum testnet. +11. If both the smart contracts run in the testnet and produce the same result - the concept is proved. + +The above process is automated and can be reproduced on any machine. + +#### Expected obstacles: +For efficiency in EVM higher order bits of types narrower than 256 bits, e.g. uint24 may be ignored or cleaned shortly before writing them to memory or before comparisons. This means, before comparison or saving higher order bits must be “manually” cleaned. + +In the Move language the code is divided into scripts and Modules. When scripts are compiled to the bytecode they become straightforward opcodes, however, modules can be recursively called from the scripts or from the other modules. Thus, modules deserialisation is much more complex and also requires recursiveness to retrieve the content of all the functions or data stored in different parts of the stack. + +#### Further development +We cannot get rid of the target language dependency, however, the source language can be abstracted to an API protocol able to call predefined smart contracts in supported languages (Move, Solidity and Rust) from other programming languages. This may initially seem to be a limitation, but the number of actually used smart contracts is not infinite. If all or nearly all known smart contract patterns are present in the template library it will give more freedoms than limitations. + +We’re planning to elaborate the most popular smart contract bytecode templates in 3 languages: Move, Solidity and Rust and write APIs for compiling and sending the resulting bytecodes to the supported blockchains. + +The smart contract templates are a temporary solution. Eventually we will add the flexibility to directly translate smart contracts from one language to another without predefined templates. + +### Ecosystem Fit + +- The VM Hub will eventually develop to become programming language independent and will enable yet nonexistent languages to seamlessly integrate with the established blockchains via Polkadot. + +## Team + +### Team members + +- Dmitry Brook - CTO, Project Lead. +- Virbal Kint - Expert in Move, Rust & Co-Founder + +### Contact + +- **Contact Name:** Dmitry Briukhanov + +- **Contact Email:** dima@xp.network.com + +- [XP.network website](https://xp.network/) + +### Legal Structure + +- **Registered Address:** HaHAgana, 15, Or Yehuda, Israel +- **Registered Legal Entity:** XP Network + +### Team Experience + +**Dmitry Briukhanov** + +- Over 18 years of experience in Development and Management +- Former Senior Software Developer in Best Systems, Israel +- Former Team Lead in NLK, Russia + +**Verbal Kint** + +- Over 5 years of experiences in Blockchain Development 8 years in Rust, 1 year in Move and 15 years of development experience overall + + +### Team Code Repos (PoC) +[XP.network repo](https://github.com/xp-network/) + +[Move Compiler](https://github.com/xp-network/move-compiler) + +[Solidity Compiler](https://github.com/xp-network/solidity-compiler) + +[EVM to Assembler](https://github.com/xp-network/evm-asm) + +[VM Hub Pallet](https://github.com/xp-network/vm_hub_pallet) + +[XP Compiler](https://github.com/xp-network/xp-compiler) + + +### Team LinkedIn Profiles + +[Dmitry Bryukhanov](https://www.linkedin.com/in/dmitry-briukhanov-60b2ab45/) + +## Development Roadmap + +### Overview + +- **Total Estimated Duration:** 6 months. +- **Total Effort:** 240 days. +- **Total Costs:** BTC 0.54 + +### Milestone 1 - VM Hub pallet Move Code to Solidity bytecode (MVP) + +- **Estimated Duration:** 20 working days (1 month) +- Working days **x** ppl. **:** 20 **x** 2 +- Effort: 40 days +- **Costs:** BTC 0.09 + +| Number | Deliverable | Specification | +| ------------- | ------------- | ------------- | +| 0. | Documentation | Documents containing product architecture as well as usage manuals | +| 1. | Bytecode deserialiser | Developing the Move and Solidity bytecode deserializers to opcode | +| 2. | Assembler parser | Developing opcode parser for Move and generating “intentions” | +| 3. | Intentions callers | Writing functions to be called with the “intentions” Move - Solidity | +| 4. | Bytecode compiler | Development of the Solidity opcode mapper to bytecode | +| 5. | Testing in Solidity Testnet| Running the generated bytecode in Ethereum testnet | + + +### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode + +- **Estimated Duration:** 34 working days (1.5 months) +- Working days **x** ppl. **:** 34 **x** 2 +- Effort: 68 days +- **Costs:** BTC 0.15 + +| Number | Deliverable | Specification | +| ------------- | ------------- | ------------- | +| 0. | Assembler Parser | Developing the parser for the Solidity opcode | +| 1. | Intentions callers | Writing functions to be called with the “intentions” Solidity - Move | +| 2. | Bytecode compiler | Development of the Move opcode mapper to Move bytecode | +| 4. | Move NFT Libraries | Development of NFT libraries in Move | +| 5. | Protocol development | Development of XP Relay Chain Protocol | +| 6. | Testing in Westend | Testing the message protocol between two pallets (Move & Solidity) messaging protocol | +| 7. | Testing in Diem Testnet| Running the generated Move bytecode in the Diem testnet | + +### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode + +- **Estimated Duration:** 33 working days (1.5 months) +- Working days **x** ppl. **:** 33 **x** 2 +- Effort: 66 days +- **Costs:** BTC 0.15 + +| Number | Deliverable | Specification | +| ------------- | ------------- | ------------- | +| 1. | Bytecode deserialiser | Developing the Rust bytecode deserializer to assembler | +| 2. | Assembler parser | Developing opcode parser for Rust and generating “intentions” | +| 3. | Intentions callers | Writing functions to be called with the “intentions” Move, Solidity - Rust and vice versa | +| 4. | Bytecode compiler | Development of the Rust opcode mapper to bytecode | +| 5. | Testing Polkadot Westnet| Running the generated from the messaging protocol bytecode in Polkadot Westnet | + +### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch + +- **Estimated Duration:** 33 working days (1.5 months) +- Working days **x** ppl. **:** 33 **x** 2 +- Effort: 66 days +- **Costs:** BTC 0.15 + +| Number | Deliverable | Specification | +| ------------- | ------------- | ------------- | +| 0. | Smart Contract templates | Writing optcode smart contract templates for known use cases in Move, Solidity, Rust | +| 1. | Integrating with Polkadot | Testing the PA in live Polkadot environment | +| 2. | Compliance Validator | Adding automated test for the above integrated modules | +| 3. | Documentation | Writing and publishing final documentation with all the amendments | +| 4. | Tutorials | Preparing and publishing tutorials with examples and exercises | +| 5. | Product Launch | Publishing the XP Network VM Hub for public use | + +## Future Plans + +**Community Plan** + +Hiring 3-5 more developers + +Joining Polkadot related events + +Publishing articles in Telegram, medium.com and other channels + + +**Development Plan** + +The template bytecode chunks will become more atomic to add flexibility to the bytecode compilation process. + +Eventually, we hope to enable our pallet to become completely dynamic, being able to process yet unseen non predefined smart contract patterns and convert them into valid bytecodes of a chosen smart contract language. + +Writing pallets with the XP Relay Chain protocols for the languages other than Solidity, Move and Rust. + +Development of bridges to the Avalanche, Binance, Cardano, Diem? (should regulations allow), Elrond, Heco and Solana blockchains. + +## Additional Information + +[The project repo:](https://github.com/xp-network) From 10480766407187d25f5ff04e21205dfe11dccf01 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Fri, 7 May 2021 12:57:13 +0300 Subject: [PATCH 02/20] Rename xp.network to xp.network.md --- applications/{xp.network => xp.network.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename applications/{xp.network => xp.network.md} (100%) diff --git a/applications/xp.network b/applications/xp.network.md similarity index 100% rename from applications/xp.network rename to applications/xp.network.md From 0bb9748d7be40a0ee7b94a7e3cacdb5e6103499d Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Fri, 7 May 2021 22:16:42 +0300 Subject: [PATCH 03/20] Update xp.network.md --- applications/xp.network.md | 69 +++++++++++++++++++++++--------------- 1 file changed, 42 insertions(+), 27 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index b57641c5c66..177ee2d54a1 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -180,23 +180,27 @@ The smart contract templates are a temporary solution. Eventually we will add th - **Total Estimated Duration:** 6 months. - **Total Effort:** 240 days. -- **Total Costs:** BTC 0.54 +- **Total Costs:** $ 30,000 ### Milestone 1 - VM Hub pallet Move Code to Solidity bytecode (MVP) - **Estimated Duration:** 20 working days (1 month) - Working days **x** ppl. **:** 20 **x** 2 - Effort: 40 days -- **Costs:** BTC 0.09 +- **Costs:** $5000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 0. | Documentation | Documents containing product architecture as well as usage manuals | -| 1. | Bytecode deserialiser | Developing the Move and Solidity bytecode deserializers to opcode | -| 2. | Assembler parser | Developing opcode parser for Move and generating “intentions” | -| 3. | Intentions callers | Writing functions to be called with the “intentions” Move - Solidity | -| 4. | Bytecode compiler | Development of the Solidity opcode mapper to bytecode | -| 5. | Testing in Solidity Testnet| Running the generated bytecode in Ethereum testnet | +| 0a. | License | Apache 2.0 | +| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | +| 0c. | Delivery time | Mid June | +| 0d. | Documentation | Documents containing product architecture as well as usage manuals | +| 1. | Smart Contract templates | 20 optcode smart contract templates for known use cases in Move, Solidity, Rust | +| 2. | Bytecode deserialiser (Move) | 20 Move and 20 Solidity smart contract bytecode templates are deserialized to opcode | +| 3. | Assembler parser (for Move) | 20 Move Opcodes are parced and “intentions” are generated | +| 4. | Intentions callers (for Move) | 20 functions to be called with the “intentions” from Move in Solidity | +| 5. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped mapper to bytecode | +| 6. | Testing in Solidity Testnet| 20 smart contract generated bytecodes are tested in Ethereum testnet | ### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode @@ -204,48 +208,59 @@ The smart contract templates are a temporary solution. Eventually we will add th - **Estimated Duration:** 34 working days (1.5 months) - Working days **x** ppl. **:** 34 **x** 2 - Effort: 68 days -- **Costs:** BTC 0.15 +- **Costs:** $8,500 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 0. | Assembler Parser | Developing the parser for the Solidity opcode | -| 1. | Intentions callers | Writing functions to be called with the “intentions” Solidity - Move | -| 2. | Bytecode compiler | Development of the Move opcode mapper to Move bytecode | -| 4. | Move NFT Libraries | Development of NFT libraries in Move | -| 5. | Protocol development | Development of XP Relay Chain Protocol | -| 6. | Testing in Westend | Testing the message protocol between two pallets (Move & Solidity) messaging protocol | -| 7. | Testing in Diem Testnet| Running the generated Move bytecode in the Diem testnet | +| 0a. | License | Apache 2.0 | +| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | +| 0c. | Delivery time | Beginning of August | +| 0d. | Documentation | Documents containing product architecture as well as user manuals | +| 1. | Assembler Parser (for Solidity) | 20 smart contracts in Solidity optcode are parced | +| 2. | Intentions callers (for Move) | 20 Move optcode templates are called with the “intentions” from Solidity optcode | +| 3. | Bytecode compiler (for Move) | 20 Move smart contract opcodes are mapped to valid Move bytecode | +| 4. | Move NFT Library | NFT library is developed in Move | +| 5. | XP Relay Chain Protocol | XP Relay Chain Protocol is developed and documented | +| 6. | Substrate Pallet| A template pellet implementing the XP Relay Chain protocol is developed | +| 7. | Testing in Diem Testnet| The generated Move bytecode tested the Diem testnet and debugged | ### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode - **Estimated Duration:** 33 working days (1.5 months) - Working days **x** ppl. **:** 33 **x** 2 - Effort: 66 days -- **Costs:** BTC 0.15 +- **Costs:** $8,250 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 1. | Bytecode deserialiser | Developing the Rust bytecode deserializer to assembler | -| 2. | Assembler parser | Developing opcode parser for Rust and generating “intentions” | -| 3. | Intentions callers | Writing functions to be called with the “intentions” Move, Solidity - Rust and vice versa | -| 4. | Bytecode compiler | Development of the Rust opcode mapper to bytecode | -| 5. | Testing Polkadot Westnet| Running the generated from the messaging protocol bytecode in Polkadot Westnet | +| 0a. | License | Apache 2.0 | +| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | +| 0c. | Delivery time |Beginning of September | +| 0d. | Documentation | Documents containing product architecture as well as user manuals | +| 1. | Bytecode deserialiser (for Rust) | Rust bytecode deserializer to assembler | +| 2. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” | +| 3. | Intentions callers | 20 opcode templates to be called with the “intentions” Move, Solidity - Rust and vice versa | +| 4. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | +| 5. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | +| 6. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | ### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch - **Estimated Duration:** 33 working days (1.5 months) - Working days **x** ppl. **:** 33 **x** 2 - Effort: 66 days -- **Costs:** BTC 0.15 +- **Costs:** $8,250 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 0. | Smart Contract templates | Writing optcode smart contract templates for known use cases in Move, Solidity, Rust | -| 1. | Integrating with Polkadot | Testing the PA in live Polkadot environment | -| 2. | Compliance Validator | Adding automated test for the above integrated modules | +| 0a. | License | Apache 2.0 | +| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | +| 0c. | Delivery time |Mid of October | +| 0d. | Documentation | Documents containing product architecture as well as user manuals | +| 1. | Integrating with Polkadot | Testing the XP Network Protocol in live Polkadot environment | +| 2. | Compliance Validator | Adding automated tests for the above integrated modules | | 3. | Documentation | Writing and publishing final documentation with all the amendments | | 4. | Tutorials | Preparing and publishing tutorials with examples and exercises | -| 5. | Product Launch | Publishing the XP Network VM Hub for public use | ## Future Plans From e74ef3e17877660cbe013b5623bd561f45b33f75 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 09:53:01 +0300 Subject: [PATCH 04/20] Update xp.network.md --- applications/xp.network.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 177ee2d54a1..1fbb64812e0 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -195,12 +195,11 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time | Mid June | | 0d. | Documentation | Documents containing product architecture as well as usage manuals | -| 1. | Smart Contract templates | 20 optcode smart contract templates for known use cases in Move, Solidity, Rust | -| 2. | Bytecode deserialiser (Move) | 20 Move and 20 Solidity smart contract bytecode templates are deserialized to opcode | -| 3. | Assembler parser (for Move) | 20 Move Opcodes are parced and “intentions” are generated | -| 4. | Intentions callers (for Move) | 20 functions to be called with the “intentions” from Move in Solidity | -| 5. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped mapper to bytecode | -| 6. | Testing in Solidity Testnet| 20 smart contract generated bytecodes are tested in Ethereum testnet | +| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Move, Solidity, Rust | +| 2. | Assembler parser (for Move) | 20 Move Opcodes are parced and “intentions” are generated for calling templates in Rust or Solidity | +| 3. | Intentions callers (for Move) | 20 functions to be called with the “intentions” from Move in Solidity | +| 4. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped to bytecode | +| 5. | Testing in Solidity Testnet| 20 Solidity smart contract bytecodes generated by step 4 are tested in Ethereum testnet and debugged | ### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode @@ -221,7 +220,7 @@ The smart contract templates are a temporary solution. Eventually we will add th | 3. | Bytecode compiler (for Move) | 20 Move smart contract opcodes are mapped to valid Move bytecode | | 4. | Move NFT Library | NFT library is developed in Move | | 5. | XP Relay Chain Protocol | XP Relay Chain Protocol is developed and documented | -| 6. | Substrate Pallet| A template pellet implementing the XP Relay Chain protocol is developed | +| 6. | Substrate Pallet (for Move & Solidity) | 2 template pallets implementing the XP Relay Chain protocol are developed | | 7. | Testing in Diem Testnet| The generated Move bytecode tested the Diem testnet and debugged | ### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode @@ -241,8 +240,9 @@ The smart contract templates are a temporary solution. Eventually we will add th | 2. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” | | 3. | Intentions callers | 20 opcode templates to be called with the “intentions” Move, Solidity - Rust and vice versa | | 4. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | -| 5. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | -| 6. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | +| 5. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | +| 6. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | +| 7. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | ### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch From f704b6e65e0c42c4a07fc7272417811164d3ec34 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 10:30:26 +0300 Subject: [PATCH 05/20] Update xp.network.md --- applications/xp.network.md | 40 +++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 1fbb64812e0..b0192999688 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -194,8 +194,8 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0a. | License | Apache 2.0 | | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time | Mid June | -| 0d. | Documentation | Documents containing product architecture as well as usage manuals | -| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Move, Solidity, Rust | +| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | +| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Move & Solidity | | 2. | Assembler parser (for Move) | 20 Move Opcodes are parced and “intentions” are generated for calling templates in Rust or Solidity | | 3. | Intentions callers (for Move) | 20 functions to be called with the “intentions” from Move in Solidity | | 4. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped to bytecode | @@ -214,7 +214,7 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0a. | License | Apache 2.0 | | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time | Beginning of August | -| 0d. | Documentation | Documents containing product architecture as well as user manuals | +| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | | 1. | Assembler Parser (for Solidity) | 20 smart contracts in Solidity optcode are parced | | 2. | Intentions callers (for Move) | 20 Move optcode templates are called with the “intentions” from Solidity optcode | | 3. | Bytecode compiler (for Move) | 20 Move smart contract opcodes are mapped to valid Move bytecode | @@ -235,14 +235,15 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0a. | License | Apache 2.0 | | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time |Beginning of September | -| 0d. | Documentation | Documents containing product architecture as well as user manuals | -| 1. | Bytecode deserialiser (for Rust) | Rust bytecode deserializer to assembler | -| 2. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” | -| 3. | Intentions callers | 20 opcode templates to be called with the “intentions” Move, Solidity - Rust and vice versa | -| 4. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | -| 5. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | -| 6. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | -| 7. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | +| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | +| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Rust | +| 2. | Bytecode deserialiser (for Rust) | Rust bytecode deserializer to assembler | +| 3. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” | +| 4. | Intentions callers | 20 opcode templates to be called with the “intentions” Move, Solidity - Rust and vice versa | +| 5. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | +| 6. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | +| 7. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | +| 8. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | ### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch @@ -256,11 +257,18 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0a. | License | Apache 2.0 | | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time |Mid of October | -| 0d. | Documentation | Documents containing product architecture as well as user manuals | -| 1. | Integrating with Polkadot | Testing the XP Network Protocol in live Polkadot environment | -| 2. | Compliance Validator | Adding automated tests for the above integrated modules | -| 3. | Documentation | Writing and publishing final documentation with all the amendments | -| 4. | Tutorials | Preparing and publishing tutorials with examples and exercises | +| 0d. | Documentation | Documents containing product architecture as well as profound user manuals | +| 1. | Integrating with Polkadot | Testing and debugging the XP Network Protocol in live Polkadot environment | +| 2. | Compliance Validator | Adding automated tests: 1. pallets (Move, Rust, Solidity), +2. XP.network protocol interactions between pallets: ++ Move -> Solidity, ++ Solidity -> Move, ++ Move -> Rust, ++ Rust -> Move, ++ Solidity-> Rust ++ Rust -> Solidity | +| 3. | Documentation | Writing and publishing final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets (for Move, Rust & Solidity)| +| 4. | Tutorials | Preparing and publishing tutorials with examples and exercises 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | ## Future Plans From 8d3e897c46b0b12dd3e8e49248007df10fb0c695 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 10:31:25 +0300 Subject: [PATCH 06/20] Update xp.network.md --- applications/xp.network.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index b0192999688..6c5d30b5f11 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -259,14 +259,7 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0c. | Delivery time |Mid of October | | 0d. | Documentation | Documents containing product architecture as well as profound user manuals | | 1. | Integrating with Polkadot | Testing and debugging the XP Network Protocol in live Polkadot environment | -| 2. | Compliance Validator | Adding automated tests: 1. pallets (Move, Rust, Solidity), -2. XP.network protocol interactions between pallets: -+ Move -> Solidity, -+ Solidity -> Move, -+ Move -> Rust, -+ Rust -> Move, -+ Solidity-> Rust -+ Rust -> Solidity | +| 2. | Compliance Validator | Adding automated tests: 1. pallets (Move, Rust, Solidity), 2. XP.network protocol interactions between pallets: Move -> Solidity, Solidity -> Move, Move -> Rust, Rust -> Move, Solidity-> Rust, Rust -> Solidity | | 3. | Documentation | Writing and publishing final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets (for Move, Rust & Solidity)| | 4. | Tutorials | Preparing and publishing tutorials with examples and exercises 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | From ba9b98d4db17d89bd5d4849156a9f3f2bd51544a Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 11:08:45 +0300 Subject: [PATCH 07/20] Update xp.network.md --- applications/xp.network.md | 41 ++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 6c5d30b5f11..cf657178f66 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -195,11 +195,11 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time | Mid June | | 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Move & Solidity | -| 2. | Assembler parser (for Move) | 20 Move Opcodes are parced and “intentions” are generated for calling templates in Rust or Solidity | -| 3. | Intentions callers (for Move) | 20 functions to be called with the “intentions” from Move in Solidity | -| 4. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped to bytecode | -| 5. | Testing in Solidity Testnet| 20 Solidity smart contract bytecodes generated by step 4 are tested in Ethereum testnet and debugged | +| 1. | Smart Contract templates | We'll write 20 optcode smart contract templates for known use cases in Move & Solidity | +| 2. | Assembler parser (for Move) | 20 Move initiating opcodes are parced and “intentions” are generated for calling the 20 templates in Solidity | +| 3. | Intentions callers (for Move) | 20 smart contracts templates are called in Solidity opcode from the “intentions” originating from Move | +| 4. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped to bytecode valid for EVM | +| 5. | Testing in Solidity Testnet| 20 Solidity smart contract bytecodes generated by step 4 are tested and debugged with Ethereum testnet till our smart contracts run without errors and the required resources are monted and burned in the blockchain | ### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode @@ -215,13 +215,13 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time | Beginning of August | | 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Assembler Parser (for Solidity) | 20 smart contracts in Solidity optcode are parced | +| 1. | Assembler Parser (for Solidity) | 20 initiating smart contracts in Solidity optcode are parced | | 2. | Intentions callers (for Move) | 20 Move optcode templates are called with the “intentions” from Solidity optcode | | 3. | Bytecode compiler (for Move) | 20 Move smart contract opcodes are mapped to valid Move bytecode | | 4. | Move NFT Library | NFT library is developed in Move | | 5. | XP Relay Chain Protocol | XP Relay Chain Protocol is developed and documented | | 6. | Substrate Pallet (for Move & Solidity) | 2 template pallets implementing the XP Relay Chain protocol are developed | -| 7. | Testing in Diem Testnet| The generated Move bytecode tested the Diem testnet and debugged | +| 7. | Testing in Diem Testnet| The generated Move bytecode is tested in the Diem testnet and debugged till it runns without errors and the expected modules and resources are added to the blockchain | ### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode @@ -236,14 +236,16 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time |Beginning of September | | 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Smart Contract templates | Write 20 optcode smart contract templates for known use cases in Rust | -| 2. | Bytecode deserialiser (for Rust) | Rust bytecode deserializer to assembler | -| 3. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” | -| 4. | Intentions callers | 20 opcode templates to be called with the “intentions” Move, Solidity - Rust and vice versa | -| 5. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | -| 6. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | -| 7. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet | -| 8. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol | +| 1. | Smart Contract templates | We'll write 20 optcode smart contract templates for known use cases in Rust | +| 2. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” from the parsed opcode | +| 3. | Intentions callers | 20 Rust opcode templates to be called with the “intentions” originating in Move | +| 4. | Intentions callers | 20 Rust opcode templates to be called with the “intentions” originating in Solidity | +| 5. | Intentions callers | 20 Move opcode templates to be called with the “intentions” originating in Rust | +| 6. | Intentions callers | 20 Solidity opcode templates to be called with the “intentions” originating in Rust | +| 7. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | +| 8. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | +| 9. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet till the transaction executes and the result is added to the blockchain | +| 10. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol till they interact according to the protocol assuming no bad actor interrupt the normal flow | ### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch @@ -258,10 +260,11 @@ The smart contract templates are a temporary solution. Eventually we will add th | 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | | 0c. | Delivery time |Mid of October | | 0d. | Documentation | Documents containing product architecture as well as profound user manuals | -| 1. | Integrating with Polkadot | Testing and debugging the XP Network Protocol in live Polkadot environment | -| 2. | Compliance Validator | Adding automated tests: 1. pallets (Move, Rust, Solidity), 2. XP.network protocol interactions between pallets: Move -> Solidity, Solidity -> Move, Move -> Rust, Rust -> Move, Solidity-> Rust, Rust -> Solidity | -| 3. | Documentation | Writing and publishing final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets (for Move, Rust & Solidity)| -| 4. | Tutorials | Preparing and publishing tutorials with examples and exercises 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | +| 1. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol till they interact according to the protocol assuming a bad actor does interrupt the normal flow or errors occur in one of the pallets | +| 2. | Integrating with Polkadot | Testing and debugging the XP Network Protocol in live Polkadot environment | +| 3. | Compliance Validator | Adding automated tests for: 1. pallets (Move, Rust, Solidity), 2. XP.network protocol interactions between pallets: Move -> Solidity, Solidity -> Move, Move -> Rust, Rust -> Move, Solidity-> Rust, Rust -> Solidity | +| 4. | Documentation | Writing and publishing final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets (for Move, Rust & Solidity)| +| 5. | Tutorials | Preparing and publishing tutorials with examples and exercises 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain (for Move, Rust & Solidity)| ## Future Plans From ff5e7c4fcb88ea1b7205dc723a6f65d654b8c2c2 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 12:32:58 +0300 Subject: [PATCH 08/20] Update xp.network.md --- applications/xp.network.md | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index cf657178f66..c56a34dc679 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -10,25 +10,32 @@ XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP.network allows non-developers to build their NFT marketplaces, galleries, museums and many more use-cases without coding. -In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets. +In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets, that can be attached to and be used by any parachain. -Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID". It will be a group of pallets, each acting like a “post office” from a post office network. They will all use our XP Relay Chain protocol. +Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID", which is especially useful when multiple transactions are executed between two blockchains. It will be a group of pallets, each acting like a “post office” in a post office network. They will all be using our XP Relay Chain protocol. -Since different blockchains may use different smart contract languages, we are aspiring to create an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which could be further validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has finished or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. +Since different blockchains may use different smart contract languages, we are developing an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which is validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has finished or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. -Because there's no NFT library in Move's standard library we will have to write it as part of this project. This will allow +Because there's no NFT library in Move's standard library we will have to write it as part of this project. This will allow: 1. other blockchains to call NFT smart contracts in Move, 2. to run NFT smart contracts in other languages from Move. #### Integration -All the NFT based applications built by our platform will be using the XP Relay Chain protocol we're building. +Any parachain can attach our pallet and use its functionality. The pallets (one for each parahcain / parathread) will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread (currently Move, Rust and Solidity). -The pallets (one for each parahcain / parathread) will act as a “post office network” supporting the protocol that enables nodes to keep track of the “topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. +All the NFT based applications built by our codeless platform will be using the XP Relay Chain protocol we're building. ### Project Details -**XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office”. A typical message will include: +The project is comprised of 2 interdependant deliverables: + +1. The XP.network Relay Chain Protocol, which will enable parachains to communicate their smart contracts in a language agnostic "intention" format. It is designed to keep track of the "TOPIC" of negotiation. + +2. Three Substrate Pallets templates that will communicate with each other implementing the XP.network protocol and will convert their original Move, Solidity & Rust smart contract bytecodes to a language agnostic "intention". This "intention" will be sent as payload in a Relay Chain callback to the target XP.network pallet attached to a designated parachain. There, the "intention" will be converted to a smart contract of the target blockchain. Once the target blockchain responds, the initiating parachain gets notified according to the XP.network protocol. + + +**XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office” for its parathread. A typical message will include: ```terminal { ID: id, //required to identify that the other blockchain’s reply is related to this request, @@ -43,7 +50,7 @@ The message inside the binary payload will be structured as follows: ![img](https://github.com/xp-network/w3f_application/blob/main/XP.network.protocol.png) -The **XP.network Handshake protocol** will look like this: +The **XP.network Handshake protocol** will roughly look like this: 1. An initiating pallet sends a message with a smart contract call to a designated pallet. 2. Once the designated pallet receives the message it unpacks the blob from the payload and returns the confirmation like this: @@ -69,13 +76,13 @@ Every parachain equipped with our pallet will know how to read such incoming mes 1. Deserialize the incoming message from bytecode to optcode, 2. Transform the commands and arguments from the optcode to a smart contract in the target language, used by the blockchain it works for. 3. It will pass the generated bytecode to the blockchain for execution. -4. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via 5. Polkadot Relay Chain callbacks with the same id it got the request. -6. The pallet which initiated the transaction will receive the reply and will match the id of the reply with the id of the request. +4. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via the Polkadot Relay Chain callbacks with the same id it got the request. +6. The pallet, which initiated the transaction, will receive the reply and will match the id of the reply with the id of the request. 7. It will deserialize the bytecode of the reply and will transform it to the bytecode of its smart contract language (should it be different from the counterpart’s). A pallet implementing this protocol consists of: -1. **Bytecode Deserializer** - it receives binary input and returns a human readable opcode. +1. **Bytecode Deserializer** - it receives a smart contract bytecode as input and returns a human readable opcode. 2. **Assembly Code Converter** - it creates the following key - value pairs: + The chosen smart contract programming language. + Template number. @@ -83,11 +90,11 @@ A pallet implementing this protocol consists of: 3. **Bytecode Compiler** - it takes the key - value pairs generated at the previous stage as an input and generates the chosen smart contract language bytecode as its output. 4. **Polkadot parathread** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. -A set of pre-programmed audited code templates are ready to be populated by the arbitrary data. Once a request is received, the templates are populated with the incoming data and are instantly compiled into transaction ready bytecode. Initially there will be a limited set of ready code templates for each platform. However, new templates will be added on a permanent regular basis. Eventually most possible use cases will be available for each bridged platform. Finally, the original smart contract bytecode will be translated directly into the target language bytecode, removing all limitations. +A set of pre-programmed audited code templates are ready to be populated by the arbitrary data. Once a request is received, the templates are populated with the incoming data and are instantly compiled into transaction ready bytecode. Initially there will be a limited set of ready smart contract opcode templates for each platform, currently 20, each representing a different use case. However, new templates will be added on a permanent regular basis. Eventually most possible use cases will be available for each bridged platform. #### [PoC](https://github.com/xp-network/xp-compiler) -Before applying for the grant we have prepared GitHub repositories with the Proof of Concept. +Before applying for the grant we have prepared a GitHub repository with the Proof of Concept. Since the idea behind the VM Hub is converting one smart contract language bytecode to another we have elaborated the following steps: From 8020921d7e1e1d4dcc0a993ed1a03e1adf5ed65c Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 13:32:33 +0300 Subject: [PATCH 09/20] Update xp.network.md --- applications/xp.network.md | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index c56a34dc679..780523a6c94 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -12,9 +12,9 @@ XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets, that can be attached to and be used by any parachain. -Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID", which is especially useful when multiple transactions are executed between two blockchains. It will be a group of pallets, each acting like a “post office” in a post office network. They will all be using our XP Relay Chain protocol. +Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID", which is especially useful when multiple transactions are executed between two blockchains. It will be a group of pallets, each acting like a “post office” in a post office network, all implementing our XP Relay Chain protocol. -Since different blockchains may use different smart contract languages, we are developing an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which is validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has finished or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. +Since different blockchains may use different smart contract languages, we are developing an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which is validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has executed or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. Because there's no NFT library in Move's standard library we will have to write it as part of this project. This will allow: 1. other blockchains to call NFT smart contracts in Move, @@ -22,17 +22,17 @@ Because there's no NFT library in Move's standard library we will have to write #### Integration -Any parachain can attach our pallet and use its functionality. The pallets (one for each parahcain / parathread) will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread (currently Move, Rust and Solidity). +Any parachain can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. We're currently working on pallets for three smart contract languages: Move, Rust and Solidity. All the NFT based applications built by our codeless platform will be using the XP Relay Chain protocol we're building. ### Project Details -The project is comprised of 2 interdependant deliverables: +The project is comprised of 2 large interdependant deliverables: 1. The XP.network Relay Chain Protocol, which will enable parachains to communicate their smart contracts in a language agnostic "intention" format. It is designed to keep track of the "TOPIC" of negotiation. -2. Three Substrate Pallets templates that will communicate with each other implementing the XP.network protocol and will convert their original Move, Solidity & Rust smart contract bytecodes to a language agnostic "intention". This "intention" will be sent as payload in a Relay Chain callback to the target XP.network pallet attached to a designated parachain. There, the "intention" will be converted to a smart contract of the target blockchain. Once the target blockchain responds, the initiating parachain gets notified according to the XP.network protocol. +2. Three Substrate Pallet templates that will communicate with each other implementing the XP.network protocol and will convert their original Move, Solidity & Rust smart contract bytecodes to a language agnostic "intention". This "intention" will be sent as payload in a Relay Chain callback to the target XP.network pallet attached to a designated parachain. There, the "intention" will be converted to a smart contract of the target blockchain. Once the target blockchain responds, the initiating parachain gets notified according to the XP.network protocol. **XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office” for its parathread. A typical message will include: @@ -73,22 +73,20 @@ The **XP.network Handshake protocol** will roughly look like this: Every parachain equipped with our pallet will know how to read such incoming messages. If the message is related to the blockchain this pallet is attached to, it will do the following: -1. Deserialize the incoming message from bytecode to optcode, -2. Transform the commands and arguments from the optcode to a smart contract in the target language, used by the blockchain it works for. -3. It will pass the generated bytecode to the blockchain for execution. -4. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via the Polkadot Relay Chain callbacks with the same id it got the request. -6. The pallet, which initiated the transaction, will receive the reply and will match the id of the reply with the id of the request. -7. It will deserialize the bytecode of the reply and will transform it to the bytecode of its smart contract language (should it be different from the counterpart’s). +1. Transform the commands and arguments from the "intention" to a smart contract in the target language, used by the blockchain it works for. +2. It will pass the generated bytecode to the blockchain for execution. +3. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via the Polkadot Relay Chain callbacks with the same id it got the request. +4. The pallet, which initiated the transaction, will receive the reply and will match the id of the reply with the id of the request. +5. It will deserialize the bytecode of the reply and will transform it to the bytecode of its smart contract language (should it be different from the counterpart’s). A pallet implementing this protocol consists of: 1. **Bytecode Deserializer** - it receives a smart contract bytecode as input and returns a human readable opcode. 2. **Assembly Code Converter** - it creates the following key - value pairs: - + The chosen smart contract programming language. - + Template number. + + Target template number. + Data to populate the smart contract with. 3. **Bytecode Compiler** - it takes the key - value pairs generated at the previous stage as an input and generates the chosen smart contract language bytecode as its output. -4. **Polkadot parathread** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. +4. **Polkadot pallet** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. A set of pre-programmed audited code templates are ready to be populated by the arbitrary data. Once a request is received, the templates are populated with the incoming data and are instantly compiled into transaction ready bytecode. Initially there will be a limited set of ready smart contract opcode templates for each platform, currently 20, each representing a different use case. However, new templates will be added on a permanent regular basis. Eventually most possible use cases will be available for each bridged platform. @@ -114,7 +112,10 @@ Since the idea behind the VM Hub is converting one smart contract language bytec The above process is automated and can be reproduced on any machine. -#### Expected obstacles: +#### The XP.network Hub is not: +This hub is by no means a bridge between the blockchains. It completely relies on Polkadot's existing infrastructure to communicate and secure interaction between the parathreads. However, we have plans of building bridges to a number of blockchains in further projects. + +#### Expected obstacles For efficiency in EVM higher order bits of types narrower than 256 bits, e.g. uint24 may be ignored or cleaned shortly before writing them to memory or before comparisons. This means, before comparison or saving higher order bits must be “manually” cleaned. In the Move language the code is divided into scripts and Modules. When scripts are compiled to the bytecode they become straightforward opcodes, however, modules can be recursively called from the scripts or from the other modules. Thus, modules deserialisation is much more complex and also requires recursiveness to retrieve the content of all the functions or data stored in different parts of the stack. From 4fef5d6670da468683fb75282f29752c11a85610 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 14:47:11 +0300 Subject: [PATCH 10/20] Update xp.network.md --- applications/xp.network.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 780523a6c94..3c6150129ae 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -46,7 +46,7 @@ Payload: blob // A binary representation of the "intenti } ``` -The message inside the binary payload will be structured as follows: +The runtime storage & the message inside the binary payload will be structured as follows: ![img](https://github.com/xp-network/w3f_application/blob/main/XP.network.protocol.png) From 422c27da760259b7a7f28685360c648fc47cb138 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 14:57:17 +0300 Subject: [PATCH 11/20] Update xp.network.md --- applications/xp.network.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 3c6150129ae..2bb53ed4b12 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -46,7 +46,7 @@ Payload: blob // A binary representation of the "intenti } ``` -The runtime storage & the message inside the binary payload will be structured as follows: +The runtime [storage](https://substrate.dev/rustdocs/v3.0.0/frame_support/storage/trait.StorageValue.html#required-methods) & the message inside the binary payload will be structured as follows: ![img](https://github.com/xp-network/w3f_application/blob/main/XP.network.protocol.png) From da0d486238c0669f6a96225eec5c742538fcaeb1 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Sun, 9 May 2021 15:44:30 +0300 Subject: [PATCH 12/20] Update xp.network.md --- applications/xp.network.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 2bb53ed4b12..6fe5ed4f66b 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -22,7 +22,7 @@ Because there's no NFT library in Move's standard library we will have to write #### Integration -Any parachain can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. We're currently working on pallets for three smart contract languages: Move, Rust and Solidity. +Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. We're currently working on pallets for three smart contract languages: Move, Rust and Solidity. All the NFT based applications built by our codeless platform will be using the XP Relay Chain protocol we're building. From f1c8187fecb63d02b196e36401a0bb532c80c712 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Tue, 11 May 2021 16:19:06 +0300 Subject: [PATCH 13/20] Update xp.network.md --- applications/xp.network.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 6fe5ed4f66b..9ce6681e05e 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -165,17 +165,15 @@ The smart contract templates are a temporary solution. Eventually we will add th ### Team Code Repos (PoC) -[XP.network repo](https://github.com/xp-network/) +[PoC Documentation](https://github.com/xp-network/poc-documentation) Method names, parameters with types, return types and description. -[Move Compiler](https://github.com/xp-network/move-compiler) +[Move Compiler](https://github.com/xp-network/move-compiler) Generates human readable code in Move. -[Solidity Compiler](https://github.com/xp-network/solidity-compiler) +[Solidity Compiler](https://github.com/xp-network/solidity-compiler) Generates human redable and byte code in Solidity. -[EVM to Assembler](https://github.com/xp-network/evm-asm) +[EVM to Assembler](https://github.com/xp-network/evm-asm) Converts Solidity bytecode to opcode. -[VM Hub Pallet](https://github.com/xp-network/vm_hub_pallet) - -[XP Compiler](https://github.com/xp-network/xp-compiler) +[XP Compiler](https://github.com/xp-network/xp-compiler) PoC project. Accepts a comand in Move, or JSON and compiles bytecode in Solidity, tests it in the bytecode in Ropsten (Ethereum testnet). ### Team LinkedIn Profiles From 1a3220748d96eb08ba66c5a54e50e64bd968ec69 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Tue, 11 May 2021 17:11:55 +0300 Subject: [PATCH 14/20] Update xp.network.md --- applications/xp.network.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 9ce6681e05e..3a05057ac02 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -165,7 +165,7 @@ The smart contract templates are a temporary solution. Eventually we will add th ### Team Code Repos (PoC) -[PoC Documentation](https://github.com/xp-network/poc-documentation) Method names, parameters with types, return types and description. +[PoC Documentation](https://xp-network.github.io/poc-documentation/) Method names, parameters with types, return types and description. [Move Compiler](https://github.com/xp-network/move-compiler) Generates human readable code in Move. From 97220a45c5b3e49f5e8fec21811f68a04a813326 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Tue, 11 May 2021 18:05:49 +0300 Subject: [PATCH 15/20] Update xp.network.md --- applications/xp.network.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/applications/xp.network.md b/applications/xp.network.md index 3a05057ac02..99ca9d7ffb3 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -71,6 +71,8 @@ The **XP.network Handshake protocol** will roughly look like this: 7. Once the above message is received by the initiating pallet it will check its integrity and will pass the result to its blockchain. It will then send the same message back to the counterpart to finish negotiation on the Topic_ID. 8. IER stands for Initiator pallet error. This flag will be raised if there is a technical issue in the initiating pallet. +![img](https://github.com/xp-network/w3f_application/blob/main/XP.network%20Protocol.png) + Every parachain equipped with our pallet will know how to read such incoming messages. If the message is related to the blockchain this pallet is attached to, it will do the following: 1. Transform the commands and arguments from the "intention" to a smart contract in the target language, used by the blockchain it works for. From a1a28b194ed8329072585381f2d804041d2b8051 Mon Sep 17 00:00:00 2001 From: Dmitry Bruhanov Date: Fri, 14 May 2021 16:01:26 +0300 Subject: [PATCH 16/20] Update xp.network.md --- applications/xp.network.md | 226 ++++++++++--------------------------- 1 file changed, 59 insertions(+), 167 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 99ca9d7ffb3..ecf9a23d740 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -1,6 +1,6 @@ # Open Grant Proposal -- **Project Name:** XP.network VM Hub. +- **Project Name:** XP.network Relay Chain Protocol. - **Team Name:** XP.network. - **Payment Address:** BTC: bc1qdpx2e6lejre536ph0csskas888ua3pz5p4pkuj. @@ -8,39 +8,29 @@ ### Overview -XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP.network allows non-developers to build their NFT marketplaces, galleries, museums and many more use-cases without coding. +XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP.network allows non-developers to build their NFT marketplaces, galleries, museums and many more use-cases without coding. We will connect our users's applications to different blockchains via Polkadot. In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets, that can be attached to and be used by any parachain. -Since it is currently hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TOPIC ID", which is especially useful when multiple transactions are executed between two blockchains. It will be a group of pallets, each acting like a “post office” in a post office network, all implementing our XP Relay Chain protocol. - -Since different blockchains may use different smart contract languages, we are developing an automated toolbox that will communicate via the Polkadot Relay Chain and will generate in the target pallet a valid code in Move, Solidity and Rust (Ink!) which is validated and compiled to byte-code to interact with the target blockchains. After the target blockchain has executed or rejected the transaction, the information about this is packed back into the reply XP Relay Chain protocol message and is sent back to the requesting pallet for passing it to the requesting blockchain. - -Because there's no NFT library in Move's standard library we will have to write it as part of this project. This will allow: -1. other blockchains to call NFT smart contracts in Move, -2. to run NFT smart contracts in other languages from Move. +Since, using the current XCMP protocol, it is hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TopicId", which is especially useful when multiple transactions(TXs) are executed between two blockchains (Lx, Ly). Besides, when a message is sent from one parachain to another the initiating parachain is not kept updated about the progress of a transaction, while our protocol will take care of that. The XP.network Relay Chain protocol will be supported by a group of pallets, each acting like a “post office” in a post office network. #### Integration -Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “Topics”. They will also "translate" the intentions of one parachain, regardless of its smart contract language, to a smart contract in the language of the target parachain / parathread. We're currently working on pallets for three smart contract languages: Move, Rust and Solidity. - -All the NFT based applications built by our codeless platform will be using the XP Relay Chain protocol we're building. +Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “TopicIds”. Each "TopicId" represents one negotiated transaction (TX) between two parachains (Lx, Ly). ### Project Details -The project is comprised of 2 large interdependant deliverables: +The project is comprised of 2 interdependent deliverables: -1. The XP.network Relay Chain Protocol, which will enable parachains to communicate their smart contracts in a language agnostic "intention" format. It is designed to keep track of the "TOPIC" of negotiation. +1. The XP.network Relay Chain Protocol, which will enable parachains to communicate kepping track of the current state of every transaction. -2. Three Substrate Pallet templates that will communicate with each other implementing the XP.network protocol and will convert their original Move, Solidity & Rust smart contract bytecodes to a language agnostic "intention". This "intention" will be sent as payload in a Relay Chain callback to the target XP.network pallet attached to a designated parachain. There, the "intention" will be converted to a smart contract of the target blockchain. Once the target blockchain responds, the initiating parachain gets notified according to the XP.network protocol. +2. A Substrate Pallet implementing the XP.network Relay Chain protocol. **XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office” for its parathread. A typical message will include: ```terminal { ID: id, //required to identify that the other blockchain’s reply is related to this request, -CallbackFunction: funcName, // a designated Polkadot Relay Chain callback function, -CallbackArguments: [ ... ], // required for the above function, To: dest, // indicates the destination parachain / parathread, Payload: blob // A binary representation of the "intention" } @@ -48,90 +38,36 @@ Payload: blob // A binary representation of the "intenti The runtime [storage](https://substrate.dev/rustdocs/v3.0.0/frame_support/storage/trait.StorageValue.html#required-methods) & the message inside the binary payload will be structured as follows: -![img](https://github.com/xp-network/w3f_application/blob/main/XP.network.protocol.png) - -The **XP.network Handshake protocol** will roughly look like this: - -1. An initiating pallet sends a message with a smart contract call to a designated pallet. -2. Once the designated pallet receives the message it unpacks the blob from the payload and returns the confirmation like this: - - a. it swaps the Source & the Destination addresses - - b. it flips the AKN flag from 0 to 1, keeping the other flags in the off state - - c. it checks whether the template of the desired type exists and the argument list matches the requirement. In case at least one does not, the DER (destination error) and END flags are set to 1. - - c. it keeps the rest of the blob intact to prove to the sender that exactly this message was received. - - d. it crafts a Message with the same ID and sends it back for the initiator to confirm that the transaction is being processed or failed. +![img](https://github.com/xp-network/w3f_application/blob/main/xp.network%20blob.png) -4. The initiating pallet checks the integrity of the parcel and whether END or DER flags were not raised and if everything is ok will return the same message (with swapped Sender & Receiver) with the INT(egrity) flag set to 1. Otherwise the INT(egrity) flag will remain 0, but NER(nework error) flag will be raised to indicate that the message was corrupted in the transport layer. -5. Once the message with the Topic_ID arrives to the destination pallet it will check the inegrity flag and if the flag is 1 it will start processing the request. Otherwise, it will retrun the same message, acknowledging broken integrity and proving that the transaction is terminated. -6. Once the request has been processed, submited to the target blockchain and a success / failure result is received form the blockchain, it will craft a new message with the same Topic_ID setting the OK or REJ flags as well as the END flag to 1. -7. Once the above message is received by the initiating pallet it will check its integrity and will pass the result to its blockchain. It will then send the same message back to the counterpart to finish negotiation on the Topic_ID. -8. IER stands for Initiator pallet error. This flag will be raised if there is a technical issue in the initiating pallet. +The **XP.network Decision Tree**, regulating the efficiency of the data flow between the two pallets, will roughly look like this: -![img](https://github.com/xp-network/w3f_application/blob/main/XP.network%20Protocol.png) +![img](https://github.com/xp-network/w3f_application/blob/main/XP.network%20Protocol-3.png) -Every parachain equipped with our pallet will know how to read such incoming messages. If the message is related to the blockchain this pallet is attached to, it will do the following: +The above scheme is a work in progress and subject to change. -1. Transform the commands and arguments from the "intention" to a smart contract in the target language, used by the blockchain it works for. -2. It will pass the generated bytecode to the blockchain for execution. -3. Once there is a reply from the blockchain, the reply will be packed into the return message of the present protocol and sent back to the requesting pallet via the Polkadot Relay Chain callbacks with the same id it got the request. -4. The pallet, which initiated the transaction, will receive the reply and will match the id of the reply with the id of the request. -5. It will deserialize the bytecode of the reply and will transform it to the bytecode of its smart contract language (should it be different from the counterpart’s). +Apart from standard setup, a pallet implementing XP.network Relay Chain Protocol consists of: -A pallet implementing this protocol consists of: +1. **Message Listener** - it listens to the incomming messages and passes them to the Decision Tree. +2. **Message Deserialiser** - it reads the contents of the binary file and populates the fields of the Message struct. +3. **Message Serialiser** - it packs the values of the Message struct into a binary representation. +4. **Message Sender** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. +5. **Runtime Storage** - it stores the binaries with the current state of the corresponding transaction. Each blob can be accessed like so: ```sender[TopicId]```. +6. **Decsision Tree** - it controlls the eficiency of the data flow between the pallets. -1. **Bytecode Deserializer** - it receives a smart contract bytecode as input and returns a human readable opcode. -2. **Assembly Code Converter** - it creates the following key - value pairs: - + Target template number. - + Data to populate the smart contract with. -3. **Bytecode Compiler** - it takes the key - value pairs generated at the previous stage as an input and generates the chosen smart contract language bytecode as its output. -4. **Polkadot pallet** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. +#### The XP.network Relay Chain protocol is not: -A set of pre-programmed audited code templates are ready to be populated by the arbitrary data. Once a request is received, the templates are populated with the incoming data and are instantly compiled into transaction ready bytecode. Initially there will be a limited set of ready smart contract opcode templates for each platform, currently 20, each representing a different use case. However, new templates will be added on a permanent regular basis. Eventually most possible use cases will be available for each bridged platform. - -#### [PoC](https://github.com/xp-network/xp-compiler) - -Before applying for the grant we have prepared a GitHub repository with the Proof of Concept. - -Since the idea behind the VM Hub is converting one smart contract language bytecode to another we have elaborated the following steps: - -1. We wrote a short smart contract in Solidity. -2. We compiled it to bytecode with the solc compiler. -3. We deserialized the bytecode into opcode. -4. An array of opcodes became a bytecode template: - a. Commands (MSTORE, SSTORE, CALLVALUE, ISZERO, JUMPI, REVERT… ) - b. Data (PUSH 80, PUSH [$] 000000000000000000000000000000000000000001, etc.) -5. We wrote a request in Move which will specify which template we want to use and will provide the data to populate the template with new data. -6. The code in Move is then compiled to bytecode & passed to our Substrate pallet. -7. The pallet deserializes the Move bytecode and extracts the instruction - which template to use and the data to populate it with. -8. The data from the request is used to populate the opcode parameters in the template. -9. The template is reassembled to bytecode. -10. We run the two bytecodes (from steps 2 & 9) in the Ethereum testnet. -11. If both the smart contracts run in the testnet and produce the same result - the concept is proved. - -The above process is automated and can be reproduced on any machine. - -#### The XP.network Hub is not: -This hub is by no means a bridge between the blockchains. It completely relies on Polkadot's existing infrastructure to communicate and secure interaction between the parathreads. However, we have plans of building bridges to a number of blockchains in further projects. - -#### Expected obstacles -For efficiency in EVM higher order bits of types narrower than 256 bits, e.g. uint24 may be ignored or cleaned shortly before writing them to memory or before comparisons. This means, before comparison or saving higher order bits must be “manually” cleaned. - -In the Move language the code is divided into scripts and Modules. When scripts are compiled to the bytecode they become straightforward opcodes, however, modules can be recursively called from the scripts or from the other modules. Thus, modules deserialisation is much more complex and also requires recursiveness to retrieve the content of all the functions or data stored in different parts of the stack. +This protocol is by no means a bridge between the blockchains. However, we have plans of building bridges to a number of blockchains in further projects. #### Further development -We cannot get rid of the target language dependency, however, the source language can be abstracted to an API protocol able to call predefined smart contracts in supported languages (Move, Solidity and Rust) from other programming languages. This may initially seem to be a limitation, but the number of actually used smart contracts is not infinite. If all or nearly all known smart contract patterns are present in the template library it will give more freedoms than limitations. - -We’re planning to elaborate the most popular smart contract bytecode templates in 3 languages: Move, Solidity and Rust and write APIs for compiling and sending the resulting bytecodes to the supported blockchains. -The smart contract templates are a temporary solution. Eventually we will add the flexibility to directly translate smart contracts from one language to another without predefined templates. +We may add more flags to reflect more states of the negotiated transaction, for example, we can specify the reasons of rejection in the target blockchain, for example, due to lack of gas. ### Ecosystem Fit -- The VM Hub will eventually develop to become programming language independent and will enable yet nonexistent languages to seamlessly integrate with the established blockchains via Polkadot. +- The XP.network protocol allows parachains to communicate in a connectionless but ordered and reliable way. The protocol allows to store the State of the negotiated TX and enables the functionality required to inform the User accordingly. + +- All it takes is to attach the XP.network pallet to a parachain. ## Team @@ -169,13 +105,9 @@ The smart contract templates are a temporary solution. Eventually we will add th ### Team Code Repos (PoC) [PoC Documentation](https://xp-network.github.io/poc-documentation/) Method names, parameters with types, return types and description. -[Move Compiler](https://github.com/xp-network/move-compiler) Generates human readable code in Move. - -[Solidity Compiler](https://github.com/xp-network/solidity-compiler) Generates human redable and byte code in Solidity. - -[EVM to Assembler](https://github.com/xp-network/evm-asm) Converts Solidity bytecode to opcode. +[XCMP Pallet example](https://github.com/xp-network/xcmp_pallet-poc/tree/master/xmessage) -[XP Compiler](https://github.com/xp-network/xp-compiler) PoC project. Accepts a comand in Move, or JSON and compiles bytecode in Solidity, tests it in the bytecode in Ropsten (Ethereum testnet). +[Protocol Blob Serialisation example](https://github.com/xp-network/serde_xp_protocol) ### Team LinkedIn Profiles @@ -186,93 +118,60 @@ The smart contract templates are a temporary solution. Eventually we will add th ### Overview -- **Total Estimated Duration:** 6 months. -- **Total Effort:** 240 days. +- **Total Estimated Duration:** 3 months. +- **Total Effort:** 120 days. - **Total Costs:** $ 30,000 -### Milestone 1 - VM Hub pallet Move Code to Solidity bytecode (MVP) +### Milestone 1 - VM Hub pallet Move bytecode to Solidity bytecode (pre-MVP) - **Estimated Duration:** 20 working days (1 month) - Working days **x** ppl. **:** 20 **x** 2 - Effort: 40 days -- **Costs:** $5000 +- **Costs:** $10,000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | | 0a. | License | Apache 2.0 | -| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | -| 0c. | Delivery time | Mid June | -| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Smart Contract templates | We'll write 20 optcode smart contract templates for known use cases in Move & Solidity | -| 2. | Assembler parser (for Move) | 20 Move initiating opcodes are parced and “intentions” are generated for calling the 20 templates in Solidity | -| 3. | Intentions callers (for Move) | 20 smart contracts templates are called in Solidity opcode from the “intentions” originating from Move | -| 4. | Bytecode compiler (for Solidity) | 20 smart contract templates in Solidity opcode are mapped to bytecode valid for EVM | -| 5. | Testing in Solidity Testnet| 20 Solidity smart contract bytecodes generated by step 4 are tested and debugged with Ethereum testnet till our smart contracts run without errors and the required resources are monted and burned in the blockchain | +| 0b. | Delivery time | Mid June | +| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 1. | XP.network Protocol | XP.network protocol is developed and documented in textual descriptions and UML diagrams | +| 2. | Message serialiser | The binary Message serialiser is built | +| 3. | Message deserialiser | The binary Message deserialiser is built | +| 4. | Runtime Storage integration | Implementing the message blob CRUD functionality in the runtime storage | +### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode (MVP) -### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode - -- **Estimated Duration:** 34 working days (1.5 months) -- Working days **x** ppl. **:** 34 **x** 2 -- Effort: 68 days -- **Costs:** $8,500 +- **Estimated Duration:** 20 working days (1 month) +- Working days **x** ppl. **:** 20 **x** 2 +- Effort: 40 days +- **Costs:** $10,000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | | 0a. | License | Apache 2.0 | -| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | -| 0c. | Delivery time | Beginning of August | -| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Assembler Parser (for Solidity) | 20 initiating smart contracts in Solidity optcode are parced | -| 2. | Intentions callers (for Move) | 20 Move optcode templates are called with the “intentions” from Solidity optcode | -| 3. | Bytecode compiler (for Move) | 20 Move smart contract opcodes are mapped to valid Move bytecode | -| 4. | Move NFT Library | NFT library is developed in Move | -| 5. | XP Relay Chain Protocol | XP Relay Chain Protocol is developed and documented | -| 6. | Substrate Pallet (for Move & Solidity) | 2 template pallets implementing the XP Relay Chain protocol are developed | -| 7. | Testing in Diem Testnet| The generated Move bytecode is tested in the Diem testnet and debugged till it runns without errors and the expected modules and resources are added to the blockchain | +| 0b. | Delivery time | Mid July | +| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 1. | Message Listener | We will develop a module listening to the messages from the Relay Chain XCMP | +| 2. | Message Sender | We will develop a module sending the messages via the Relay Chain XCMP | +| 3. | Decision Tree | We will develop the efficient data flow controllers | +| 4. | Relay Cain Integration | We will establish communication between two pallets implementing the XP.network Relay Chain protocol via the Westend Relay Chain | ### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode -- **Estimated Duration:** 33 working days (1.5 months) -- Working days **x** ppl. **:** 33 **x** 2 -- Effort: 66 days -- **Costs:** $8,250 - -| Number | Deliverable | Specification | -| ------------- | ------------- | ------------- | -| 0a. | License | Apache 2.0 | -| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | -| 0c. | Delivery time |Beginning of September | -| 0d. | Documentation | Documents containing product architecture as well as basic user manuals | -| 1. | Smart Contract templates | We'll write 20 optcode smart contract templates for known use cases in Rust | -| 2. | Assembler parser (Rust) | Developing opcode parser for Rust and generating “intentions” from the parsed opcode | -| 3. | Intentions callers | 20 Rust opcode templates to be called with the “intentions” originating in Move | -| 4. | Intentions callers | 20 Rust opcode templates to be called with the “intentions” originating in Solidity | -| 5. | Intentions callers | 20 Move opcode templates to be called with the “intentions” originating in Rust | -| 6. | Intentions callers | 20 Solidity opcode templates to be called with the “intentions” originating in Rust | -| 7. | Bytecode compiler (for Rust) | 20 Rust opcode smart contract templates are mapped to Rust bytecode | -| 8. | Substrate Pallet (for Rust) | 1 template pallet implementing the XP Relay Chain protocol is developed | -| 9. | Testing in Polkadot Westnet| Testing and debugging 20 generated smart contracts inside Rust bytecode in Polkadot Westnet till the transaction executes and the result is added to the blockchain | -| 10. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol till they interact according to the protocol assuming no bad actor interrupt the normal flow | - -### Milestone 4 — Smart Contract templates in Move, Solidity, Rust, testing and launch - -- **Estimated Duration:** 33 working days (1.5 months) -- Working days **x** ppl. **:** 33 **x** 2 -- Effort: 66 days -- **Costs:** $8,250 +- **Estimated Duration:** 20 working days (1 month) +- Working days **x** ppl. **:** 20 **x** 2 +- Effort: 40 days +- **Costs:** $10,000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | | 0a. | License | Apache 2.0 | -| 0b. | Description | We will support all functional modules that have been implemented on Polkadot. | -| 0c. | Delivery time |Mid of October | -| 0d. | Documentation | Documents containing product architecture as well as profound user manuals | -| 1. | Testing in Westend | XP Relay Chain Protocol is tested between tree pallets (Move, Rust & Solidity) messaging protocol till they interact according to the protocol assuming a bad actor does interrupt the normal flow or errors occur in one of the pallets | -| 2. | Integrating with Polkadot | Testing and debugging the XP Network Protocol in live Polkadot environment | -| 3. | Compliance Validator | Adding automated tests for: 1. pallets (Move, Rust, Solidity), 2. XP.network protocol interactions between pallets: Move -> Solidity, Solidity -> Move, Move -> Rust, Rust -> Move, Solidity-> Rust, Rust -> Solidity | -| 4. | Documentation | Writing and publishing final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets (for Move, Rust & Solidity)| -| 5. | Tutorials | Preparing and publishing tutorials with examples and exercises 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain (for Move, Rust & Solidity)| +| 0b. | Delivery time | Mid August | +| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 1. | Compliance Validator | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | +| 2. | Documentation | Writing the final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets - textual & UML| +| 3. | Tutorials | Preparing tutorials with examples 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | +| 4. | Testing in Polkadot Westnet| Testing and debugging Polkadot Westnet till 20 example transactions execute and the results are added to the blockchain | ## Future Plans @@ -284,17 +183,10 @@ Joining Polkadot related events Publishing articles in Telegram, medium.com and other channels - **Development Plan** -The template bytecode chunks will become more atomic to add flexibility to the bytecode compilation process. - -Eventually, we hope to enable our pallet to become completely dynamic, being able to process yet unseen non predefined smart contract patterns and convert them into valid bytecodes of a chosen smart contract language. - -Writing pallets with the XP Relay Chain protocols for the languages other than Solidity, Move and Rust. - -Development of bridges to the Avalanche, Binance, Cardano, Diem? (should regulations allow), Elrond, Heco and Solana blockchains. +The XP.network Relay Chain Protocol is a work in progress. Once version 1.0.0 is implemented, we're planning to keep improving the protocol, eliminating yet undiscovered bugs and adding more functionality and flexibility. -## Additional Information +## Additional Information [The project repo:](https://github.com/xp-network) From af23b7f279c501f405b7dbcf7af4a31862639b0f Mon Sep 17 00:00:00 2001 From: Dima Brook Date: Wed, 19 May 2021 17:35:38 +0300 Subject: [PATCH 17/20] Update xp.network.md --- applications/xp.network.md | 49 ++++++++++++++++++++++++++------------ 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index ecf9a23d740..726398a1f5c 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -12,17 +12,25 @@ XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets, that can be attached to and be used by any parachain. -Since, using the current XCMP protocol, it is hard to trace whether an incoming message is related to any previous transaction or request, we will elaborate a protocol that will enable such tracking by the "TopicId", which is especially useful when multiple transactions(TXs) are executed between two blockchains (Lx, Ly). Besides, when a message is sent from one parachain to another the initiating parachain is not kept updated about the progress of a transaction, while our protocol will take care of that. The XP.network Relay Chain protocol will be supported by a group of pallets, each acting like a “post office” in a post office network. +The XP.network protocol allows parachains to communicate in a connectionless but ordered and reliable way. The protocol allows to store the State of a negotiated transaction (TX) and enables the functionality required to inform the User or the original parachain accordingly. + +At the moment, XCMP is a work in progress and SPREE only exists in documentation and will only be developed after the parachains become available. With all the advantages of the two above mentioned protocols, neither of them offers the precision XP.network Protocol does. + +1. Our protocol allows parachains to mark and store the current state of every transaction in a blob with a unique TopicID. Even though the future implementations of XCMP will allow to create channels between two parachains, TopicIDs will still be useful when multiple transactions are being executed between two parachains simultaneously. Another XCMP feature - ordered delivery - does not provide the same precision as the TopicID does. The TopicID does not rely on the order of messages. It relies on the unique identifier which is the safest pointer. + +2. Once the state of a transaction changes or an error occurs, the requesting parachain gets notified. Some of such messages help keep the end user properly informed about the state of events with his/her transaction, others inform the pallets that the transaction has terminated and the memory must be freed from the blob, which might be expensive to store, especially when it is no longer needed. + +3. Another feature uses the fact that a pallet, implementing our protocol, is a part of its parent parachain. Therefore, it has no additional overhead in tracking whether the transaction succeeded or failed. It listens to events related to the transaction and notifies the requesting parachian of the result and provides the data of the outcome when it becomes available without being additionally requested about this information. #### Integration -Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “TopicIds”. Each "TopicId" represents one negotiated transaction (TX) between two parachains (Lx, Ly). +Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “TopicIDs”. Each "TopicID" represents one negotiated transaction (TX) between two parachains (Lx, Ly). ### Project Details The project is comprised of 2 interdependent deliverables: -1. The XP.network Relay Chain Protocol, which will enable parachains to communicate kepping track of the current state of every transaction. +1. The XP.network Relay Chain Protocol, which will enable parachains to communicate keeping track of the current state of every transaction. 2. A Substrate Pallet implementing the XP.network Relay Chain protocol. @@ -38,7 +46,14 @@ Payload: blob // A binary representation of the "intenti The runtime [storage](https://substrate.dev/rustdocs/v3.0.0/frame_support/storage/trait.StorageValue.html#required-methods) & the message inside the binary payload will be structured as follows: -![img](https://github.com/xp-network/w3f_application/blob/main/xp.network%20blob.png) +![img](https://github.com/xp-network/w3f_application/blob/main/xp.network_blob.png) + +Even the complete message adds only 64 additional bits to the original TX binary code. 16 bits for the TopicID, 16 bits for the flags and 32 bits for the length of the TX binary. In order to reduce the overhead: +1. When notifying about errors or about submission of the TX bytecode for execution, only the first 32 bits are attached to the message: 16 bits of the TopicID and 16 bits for flags. They will be used to update the blob, where the state of a TX is stored during the number of seconds when a TX is executed in a target parachain. The extrinsics bytecode is not moved when it is not required. +2. Should, sometime in the future, even such tiny notifications create a noticeable overhead, we will:
+ a. join them into batches to reduce the number of messages
+ b. remove the AKN notification, which is required till SPREE becomes available to confirm that the message has been received with the same TX bytecode as the sender's. + The **XP.network Decision Tree**, regulating the efficiency of the data flow between the two pallets, will roughly look like this: @@ -48,12 +63,12 @@ The above scheme is a work in progress and subject to change. Apart from standard setup, a pallet implementing XP.network Relay Chain Protocol consists of: -1. **Message Listener** - it listens to the incomming messages and passes them to the Decision Tree. +1. **Message Listener** - it listens to the incoming messages and passes them to the Decision Tree. 2. **Message Deserialiser** - it reads the contents of the binary file and populates the fields of the Message struct. 3. **Message Serialiser** - it packs the values of the Message struct into a binary representation. 4. **Message Sender** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. -5. **Runtime Storage** - it stores the binaries with the current state of the corresponding transaction. Each blob can be accessed like so: ```sender[TopicId]```. -6. **Decsision Tree** - it controlls the eficiency of the data flow between the pallets. +5. **Runtime Storage** - it stores the binaries with the current state of the corresponding transaction. Each blob can be accessed like so: ```sender[TopicID]```. +6. **Decision Tree** - it controls the efficiency of the data flow between the pallets. #### The XP.network Relay Chain protocol is not: @@ -80,13 +95,13 @@ We may add more flags to reflect more states of the negotiated transaction, for - **Contact Name:** Dmitry Briukhanov -- **Contact Email:** dima@xp.network.com +- **Contact Email:** dima@xp.network - [XP.network website](https://xp.network/) ### Legal Structure -- **Registered Address:** HaHAgana, 15, Or Yehuda, Israel +- **Registered Address:** HaHagana, 15, Or Yehuda, Israel - **Registered Legal Entity:** XP Network ### Team Experience @@ -122,7 +137,7 @@ We may add more flags to reflect more states of the negotiated transaction, for - **Total Effort:** 120 days. - **Total Costs:** $ 30,000 -### Milestone 1 - VM Hub pallet Move bytecode to Solidity bytecode (pre-MVP) +### Milestone 1 - Rust Substrate pallet (pre-MVP) - **Estimated Duration:** 20 working days (1 month) - Working days **x** ppl. **:** 20 **x** 2 @@ -134,12 +149,13 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0a. | License | Apache 2.0 | | 0b. | Delivery time | Mid June | | 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | XP.network Protocol | XP.network protocol is developed and documented in textual descriptions and UML diagrams | -| 2. | Message serialiser | The binary Message serialiser is built | -| 3. | Message deserialiser | The binary Message deserialiser is built | +| 2. | Message serializer | The binary Message serializer is built | +| 3. | Message deserializer | The binary Message deserializer is built | | 4. | Runtime Storage integration | Implementing the message blob CRUD functionality in the runtime storage | -### Milestone 2 — VM Hub pallet Solidity Code to Move bytecode (MVP) +### Milestone 2 — Sender, Litenter and the Decision tree (MVP) - **Estimated Duration:** 20 working days (1 month) - Working days **x** ppl. **:** 20 **x** 2 @@ -151,12 +167,13 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0a. | License | Apache 2.0 | | 0b. | Delivery time | Mid July | | 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | Message Listener | We will develop a module listening to the messages from the Relay Chain XCMP | | 2. | Message Sender | We will develop a module sending the messages via the Relay Chain XCMP | | 3. | Decision Tree | We will develop the efficient data flow controllers | | 4. | Relay Cain Integration | We will establish communication between two pallets implementing the XP.network Relay Chain protocol via the Westend Relay Chain | -### Milestone 3 — VM Hub pallet Move Code & Solidity bytecode to Rust bytecode +### Milestone 3 — Documentation & Release - **Estimated Duration:** 20 working days (1 month) - Working days **x** ppl. **:** 20 **x** 2 @@ -168,6 +185,7 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0a. | License | Apache 2.0 | | 0b. | Delivery time | Mid August | | 0c. | Documentation | Documents containing product architecture as well as basic user manuals | +| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | Compliance Validator | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | | 2. | Documentation | Writing the final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets - textual & UML| | 3. | Tutorials | Preparing tutorials with examples 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | @@ -185,8 +203,9 @@ Publishing articles in Telegram, medium.com and other channels **Development Plan** -The XP.network Relay Chain Protocol is a work in progress. Once version 1.0.0 is implemented, we're planning to keep improving the protocol, eliminating yet undiscovered bugs and adding more functionality and flexibility. +When the new version of XCMP is developed we will migrate our protocol there. Once SPREE becomes available we will move our protocol there, since it will provide more security in its own storage, that a parachain cannot forge or alter. ## Additional Information [The project repo:](https://github.com/xp-network) + From 21a12de9dbcd395e5176e244ad695de61aad5fc2 Mon Sep 17 00:00:00 2001 From: Dima Brook Date: Sat, 22 May 2021 08:06:01 +0300 Subject: [PATCH 18/20] Update xp.network.md --- applications/xp.network.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 726398a1f5c..b07a015e682 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -1,6 +1,6 @@ # Open Grant Proposal -- **Project Name:** XP.network Relay Chain Protocol. +- **Project Name:** XP.network Protocol. - **Team Name:** XP.network. - **Payment Address:** BTC: bc1qdpx2e6lejre536ph0csskas888ua3pz5p4pkuj. @@ -30,12 +30,12 @@ Any parachain, parathread or bridge can attach our pallet and use its functional The project is comprised of 2 interdependent deliverables: -1. The XP.network Relay Chain Protocol, which will enable parachains to communicate keeping track of the current state of every transaction. +1. The XP.network Protocol, which will enable parachains to communicate keeping track of the current state of every transaction. -2. A Substrate Pallet implementing the XP.network Relay Chain protocol. +2. A Substrate Pallet implementing the XP.network protocol. -**XP Relay Chain Protocol** will be supported by a number of pallets, each acting as a “post office” for its parathread. A typical message will include: +**XP Protocol** will be supported by a number of pallets, each acting as a “post office” for its parathread. A typical message will include: ```terminal { ID: id, //required to identify that the other blockchain’s reply is related to this request, @@ -57,20 +57,20 @@ Even the complete message adds only 64 additional bits to the original TX binary The **XP.network Decision Tree**, regulating the efficiency of the data flow between the two pallets, will roughly look like this: -![img](https://github.com/xp-network/w3f_application/blob/main/XP.network%20Protocol-3.png) +![img](https://github.com/xp-network/w3f_application/blob/main/XP.network%20Protocol-2.png) The above scheme is a work in progress and subject to change. -Apart from standard setup, a pallet implementing XP.network Relay Chain Protocol consists of: +Apart from standard setup, a pallet implementing XP.network Protocol consists of: 1. **Message Listener** - it listens to the incoming messages and passes them to the Decision Tree. 2. **Message Deserialiser** - it reads the contents of the binary file and populates the fields of the Message struct. 3. **Message Serialiser** - it packs the values of the Message struct into a binary representation. -4. **Message Sender** - it uses the Relay Chain callback mechanism to communicate with the other parachains and parathreads using XP Network protocol. +4. **Message Sender** - it uses the XCMP to communicate with the other parachains and parathreads using XP Network protocol. 5. **Runtime Storage** - it stores the binaries with the current state of the corresponding transaction. Each blob can be accessed like so: ```sender[TopicID]```. 6. **Decision Tree** - it controls the efficiency of the data flow between the pallets. -#### The XP.network Relay Chain protocol is not: +#### The XP.network Protocol is not: This protocol is by no means a bridge between the blockchains. However, we have plans of building bridges to a number of blockchains in further projects. @@ -151,8 +151,8 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0c. | Documentation | Documents containing product architecture as well as basic user manuals | | 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | XP.network Protocol | XP.network protocol is developed and documented in textual descriptions and UML diagrams | -| 2. | Message serializer | The binary Message serializer is built | -| 3. | Message deserializer | The binary Message deserializer is built | +| 2. | Message serializer | The binary Message serializer is built with SCALE | +| 3. | Message deserializer | The binary Message deserializer is built with SCALE | | 4. | Runtime Storage integration | Implementing the message blob CRUD functionality in the runtime storage | ### Milestone 2 — Sender, Litenter and the Decision tree (MVP) @@ -168,10 +168,10 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0b. | Delivery time | Mid July | | 0c. | Documentation | Documents containing product architecture as well as basic user manuals | | 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | -| 1. | Message Listener | We will develop a module listening to the messages from the Relay Chain XCMP | -| 2. | Message Sender | We will develop a module sending the messages via the Relay Chain XCMP | +| 1. | Message Listener | We will develop a module listening to the messages from the XCMP | +| 2. | Message Sender | We will develop a module sending the messages via XCMP | | 3. | Decision Tree | We will develop the efficient data flow controllers | -| 4. | Relay Cain Integration | We will establish communication between two pallets implementing the XP.network Relay Chain protocol via the Westend Relay Chain | +| 4. | Relay Cain Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | ### Milestone 3 — Documentation & Release @@ -189,7 +189,7 @@ We may add more flags to reflect more states of the negotiated transaction, for | 1. | Compliance Validator | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | | 2. | Documentation | Writing the final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets - textual & UML| | 3. | Tutorials | Preparing tutorials with examples 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | -| 4. | Testing in Polkadot Westnet| Testing and debugging Polkadot Westnet till 20 example transactions execute and the results are added to the blockchain | +| 4. | Testing in Kusama| Testing and debugging in Kusama till 20 example transactions execute and the results are added to the blockchain | ## Future Plans From 4adbf6552ecf3601c87f397eefed304dd6858673 Mon Sep 17 00:00:00 2001 From: Dima Brook Date: Sun, 30 May 2021 11:25:40 +0300 Subject: [PATCH 19/20] Update xp.network.md --- applications/xp.network.md | 94 ++++++++++++++++++++++---------------- 1 file changed, 54 insertions(+), 40 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index b07a015e682..17f364946ad 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -10,7 +10,7 @@ XP.network is a codeless platform for building blockchain agnostic NFT DApps. XP.network allows non-developers to build their NFT marketplaces, galleries, museums and many more use-cases without coding. We will connect our users's applications to different blockchains via Polkadot. -In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by a network of XP.network pallets, that can be attached to and be used by any parachain. +In order to synchronize communication between different [parachains](https://research.web3.foundation/en/latest/polkadot/XCMP/index.html) we want to build our own protocol which will be used by an XP.network pallet, that can be attached to and be used by any parachain. The XP.network protocol allows parachains to communicate in a connectionless but ordered and reliable way. The protocol allows to store the State of a negotiated transaction (TX) and enables the functionality required to inform the User or the original parachain accordingly. @@ -22,6 +22,7 @@ At the moment, XCMP is a work in progress and SPREE only exists in documentation 3. Another feature uses the fact that a pallet, implementing our protocol, is a part of its parent parachain. Therefore, it has no additional overhead in tracking whether the transaction succeeded or failed. It listens to events related to the transaction and notifies the requesting parachian of the result and provides the data of the outcome when it becomes available without being additionally requested about this information. + #### Integration Any parachain, parathread or bridge can attach our pallet and use its functionality. The pallets will act as a “post office network” supporting the protocol that enables nodes to keep track of the “TopicIDs”. Each "TopicID" represents one negotiated transaction (TX) between two parachains (Lx, Ly). @@ -48,12 +49,34 @@ The runtime [storage](https://substrate.dev/rustdocs/v3.0.0/frame_support/storag ![img](https://github.com/xp-network/w3f_application/blob/main/xp.network_blob.png) -Even the complete message adds only 64 additional bits to the original TX binary code. 16 bits for the TopicID, 16 bits for the flags and 32 bits for the length of the TX binary. In order to reduce the overhead: +Even the complete message adds only 64 bits to the original TX binary code. 16 bits for the TopicID, 16 bits for the flags and 32 bits for the length of the TX binary. In order to reduce the overhead: 1. When notifying about errors or about submission of the TX bytecode for execution, only the first 32 bits are attached to the message: 16 bits of the TopicID and 16 bits for flags. They will be used to update the blob, where the state of a TX is stored during the number of seconds when a TX is executed in a target parachain. The extrinsics bytecode is not moved when it is not required. 2. Should, sometime in the future, even such tiny notifications create a noticeable overhead, we will:
a. join them into batches to reduce the number of messages
b. remove the AKN notification, which is required till SPREE becomes available to confirm that the message has been received with the same TX bytecode as the sender's. +Our protocol addresses two of the raised [issues](https://github.com/paritytech/polkadot/issues?q=is%3Aissue+xcm+is%3Aopen) ++ [Adding events for UMP message execution results](https://github.com/paritytech/polkadot/issues/2720) +Instead of broadcasting events spamming the network, we message only the target pallet. Reducing excessive computation in the blockchain. + ++ [XCM Retry Queue #2702](https://github.com/paritytech/polkadot/issues/2702) suggests several use-cases, where our protocol can come handy: + +1. When a nonsense message or erroneous code is sent, a DER (destination error flag) is raised, and if a parachain produces a more wordy explanation, it would be delivered in the TXData section of the blob. Since such TX should not be retried in the future the END flag will also be raised instructing the sender pallet to remove the transaction state blob from the storage. + +2. When an unknown token has been transferred to a parachain and the target parachain failed to process the request, an error flag will be raised. +An END flag will not be raised, therefore the state blob will not be removed. This will prevent the assets from being irreversibly lost and will leave the opportunity for the developers to add the token and allow the transaction to be successfully processed at a later time. + +3. In case there's not enough gas in the target parachain the process will be the same as above in point 2. + +XP.network uses XCM protocol as its transport layer and complies with the XCM protocol's four ['A's](https://github.com/paritytech/xcm-format): + +1. Asynchronous: our protocol does not block the sender in any way. + +2. Absolute: XP.network protocol informs the sending parachain that the message has been delivered by setting the AKN flag and sending back the message with the same TopicID. The order becomes insignificant, due to the fact, that the TopicID is more precise than the order. + +3. Asymmetric: The messages do not have a result. The outcome of the message processing is always delivered in a separate message. + +4. Agnostic: XP.network protocol makes no assumptions about the Consensus System of either parachains. The **XP.network Decision Tree**, regulating the efficiency of the data flow between the two pallets, will roughly look like this: @@ -137,59 +160,51 @@ We may add more flags to reflect more states of the negotiated transaction, for - **Total Effort:** 120 days. - **Total Costs:** $ 30,000 -### Milestone 1 - Rust Substrate pallet (pre-MVP) +### Milestone 1 - Alpha version of XP.network Protocol + +- **Estimated Duration:** 30 working days (1.5 month) +- Working days **x** ppl. **:** 30 **x** 2 +- Effort: 60 days +- **Costs:** $15,000 -- **Estimated Duration:** 20 working days (1 month) -- Working days **x** ppl. **:** 20 **x** 2 -- Effort: 40 days -- **Costs:** $10,000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 0a. | License | Apache 2.0 | -| 0b. | Delivery time | Mid June | -| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | -| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | +| 0a. | License | Apache 2.0 / GPLv3 / MIT / Unlicense | +| 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | +| 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | +| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | +| 0e. | Article | We will publish an **article**/workshop that explains 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain +| 0f. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | XP.network Protocol | XP.network protocol is developed and documented in textual descriptions and UML diagrams | | 2. | Message serializer | The binary Message serializer is built with SCALE | | 3. | Message deserializer | The binary Message deserializer is built with SCALE | | 4. | Runtime Storage integration | Implementing the message blob CRUD functionality in the runtime storage | +| 5. | Testing & debugging | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | +| 6. | Testing in Kusama| Testing and debugging in Kusama till 10 example transactions execute and the results are added to the blockchain | + +### Milestone 2 — The production version of XP.network Protocol -### Milestone 2 — Sender, Litenter and the Decision tree (MVP) +- **Estimated Duration:** 30 working days (1.5 month) +- Working days **x** ppl. **:** 30 **x** 2 +- Effort: 60 days +- **Costs:** $15,000 -- **Estimated Duration:** 20 working days (1 month) -- Working days **x** ppl. **:** 20 **x** 2 -- Effort: 40 days -- **Costs:** $10,000 | Number | Deliverable | Specification | | ------------- | ------------- | ------------- | -| 0a. | License | Apache 2.0 | -| 0b. | Delivery time | Mid July | -| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | -| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | +| 0a. | License | Apache 2.0 / GPLv3 / MIT / Unlicense | +| 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | +| 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | +| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | +| 0e. | Article | We will publish an **article**/workshop that explains 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain +| 0f. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | Message Listener | We will develop a module listening to the messages from the XCMP | | 2. | Message Sender | We will develop a module sending the messages via XCMP | | 3. | Decision Tree | We will develop the efficient data flow controllers | -| 4. | Relay Cain Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | - -### Milestone 3 — Documentation & Release - -- **Estimated Duration:** 20 working days (1 month) -- Working days **x** ppl. **:** 20 **x** 2 -- Effort: 40 days -- **Costs:** $10,000 - -| Number | Deliverable | Specification | -| ------------- | ------------- | ------------- | -| 0a. | License | Apache 2.0 | -| 0b. | Delivery time | Mid August | -| 0c. | Documentation | Documents containing product architecture as well as basic user manuals | -| 0d. | PSP | Drafting, Calling for feedback, Acceptance, Integration | -| 1. | Compliance Validator | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | -| 2. | Documentation | Writing the final documentation with all the amendments 1. XP.Network protocol, 2. XP.network pallets - textual & UML| -| 3. | Tutorials | Preparing tutorials with examples 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain | -| 4. | Testing in Kusama| Testing and debugging in Kusama till 20 example transactions execute and the results are added to the blockchain | +| 4. | XCMP Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | +| 5. | Testing & debugging | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | +| 6. | Testing in Kusama| Testing and debugging in Kusama till 10 example transactions execute and the results are added to the blockchain | ## Future Plans @@ -208,4 +223,3 @@ When the new version of XCMP is developed we will migrate our protocol there. On ## Additional Information [The project repo:](https://github.com/xp-network) - From bb396d14cb29c25b95c8fdf22b47744c76c8690e Mon Sep 17 00:00:00 2001 From: Dima Brook Date: Wed, 2 Jun 2021 13:38:39 +0300 Subject: [PATCH 20/20] Update xp.network.md --- applications/xp.network.md | 61 +++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/applications/xp.network.md b/applications/xp.network.md index 17f364946ad..efb1c1b95ac 100644 --- a/applications/xp.network.md +++ b/applications/xp.network.md @@ -156,16 +156,16 @@ We may add more flags to reflect more states of the negotiated transaction, for ### Overview -- **Total Estimated Duration:** 3 months. -- **Total Effort:** 120 days. -- **Total Costs:** $ 30,000 +- **Total Estimated Duration:** 1.5 months. +- **Total Effort:** 36 working days. +- **Total Costs:** $ 12,000 -### Milestone 1 - Alpha version of XP.network Protocol +### Milestone 1 - PoC of XP.network Protocol -- **Estimated Duration:** 30 working days (1.5 month) -- Working days **x** ppl. **:** 30 **x** 2 -- Effort: 60 days -- **Costs:** $15,000 +- **Estimated Duration:** 12 working days (2.5 weeks) +- Working days **x** ppl. **:** 6 **x** 2 +- Effort: 12 w/days +- **Costs:** $0,00 | Number | Deliverable | Specification | @@ -174,21 +174,21 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | | 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | | 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | -| 0e. | Article | We will publish an **article**/workshop that explains 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain +| 0e. | Article | We will publish an **article**/workshop that explains 1. What is XP.network protocol and 2. how it can be used. | | 0f. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | XP.network Protocol | XP.network protocol is developed and documented in textual descriptions and UML diagrams | | 2. | Message serializer | The binary Message serializer is built with SCALE | | 3. | Message deserializer | The binary Message deserializer is built with SCALE | | 4. | Runtime Storage integration | Implementing the message blob CRUD functionality in the runtime storage | -| 5. | Testing & debugging | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | +| 5. | Testing & debugging | Developing automated tests for the XP.network protocol BLOB serialization / Deserialization, storage, updating, deletion with min 85% code coverage | | 6. | Testing in Kusama| Testing and debugging in Kusama till 10 example transactions execute and the results are added to the blockchain | -### Milestone 2 — The production version of XP.network Protocol +### Milestone 2 — Alpha version of XP.network Protocol -- **Estimated Duration:** 30 working days (1.5 month) -- Working days **x** ppl. **:** 30 **x** 2 -- Effort: 60 days -- **Costs:** $15,000 +- **Estimated Duration:** 12 working days (2.5 weeks) +- Working days **x** ppl. **:** 6 **x** 2 +- Effort: 12 w/days +- **Costs:** $4,000 | Number | Deliverable | Specification | @@ -197,14 +197,35 @@ We may add more flags to reflect more states of the negotiated transaction, for | 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | | 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | | 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | -| 0e. | Article | We will publish an **article**/workshop that explains 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain +| 0e. | Article | We will publish an **article**/workshop that explains how is XP.network Protocol integrated in Polkadot | | 0f. | PSP | Drafting, Calling for feedback, Acceptance, Integration | | 1. | Message Listener | We will develop a module listening to the messages from the XCMP | | 2. | Message Sender | We will develop a module sending the messages via XCMP | -| 3. | Decision Tree | We will develop the efficient data flow controllers | -| 4. | XCMP Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | -| 5. | Testing & debugging | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | -| 6. | Testing in Kusama| Testing and debugging in Kusama till 10 example transactions execute and the results are added to the blockchain | +| 3. | XCMP Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | +| 4. | Testing & debugging | Developing automated tests for the XP.network protocol XCMP message processing between two pallets with min 85% code coverage | +| 5. | Testing in Kusama| Testing and debugging in Kusama till 10 example transactions execute and the results are added to the blockchain | + + +### Milestone 3 — Production version of XP.network Protocol + +- **Estimated Duration:** 12 working days (2.5 weeks) +- Working days **x** ppl. **:** 6 **x** 2 +- Effort: 12 w/days +- **Costs:** $8,000 + + +| Number | Deliverable | Specification | +| ------------- | ------------- | ------------- | +| 0a. | License | Apache 2.0 / GPLv3 / MIT / Unlicense | +| 0b. | Documentation | We will provide both **inline documentation** of the code and a basic **tutorial** that explains how a user can (for example) spin up one of our Substrate nodes and send test transactions, which will show how the new functionality works. | +| 0c. | Testing Guide | Core functions will be fully covered by unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | +| 0d. | Docker | We will provide a Dockerfile(s) that can be used to test all the functionality delivered with this milestone. | +| 0e. | Article | We will publish an **article**/workshop that explains 1. How to use XP.network Protocol, 2. How to attach an XP.Network pallet to a parachain +| 0f. | PSP | Drafting, Calling for feedback, Acceptance, Integration | +| 1. | Decision Tree | We will develop the efficient data flow controllers and finite state machine logics | +| 2. | XCMP Integration | We will establish communication between two pallets implementing the XP.network Protocol in Kusama | +| 3. | Testing & debugging | Developing automated tests for the XP.network protocol interactions between pallets with min 85% code coverage | +| 4. | Testing in Kusama| Testing and debugging in Kusama till 20 example transactions execute and the results are added to the blockchain | ## Future Plans