diff --git a/book/src/architecture/component-pipeline.md b/book/src/architecture/component-pipeline.md index eb236b7c3e5..b452c9bba33 100644 --- a/book/src/architecture/component-pipeline.md +++ b/book/src/architecture/component-pipeline.md @@ -397,7 +397,7 @@ relative to `core_based_updates/`): | Read | Asked of Core | Code | Kept in | |---|---|---|---| | Core fork activations, from protocol version 15 | The coinbase payload version of the block at `c`, until the fork is recorded | `update_core_info/v1/` | GroveDB `Misc`; see [Core Fork Activations](core-fork-activations.md) | -| Masternode list | `protx listdiff` from the previous block's `c` to `c`; also the coinbase payload version at `c` when a reported operator key may only be re-encoded | `update_masternode_list/` | Platform state, and masternode identities in GroveDB | +| Masternode list | `protx listdiff` from the previous block's `c` to `c`; up to protocol version 14 also the coinbase payload version at `c` when a reported operator key may only be re-encoded (from 15 the Core fork record answers); from protocol version 15 also `protx info` at `c` for a masternode whose operator key changed without a printed payout | `update_masternode_list/` | Platform state, and masternode identities in GroveDB | | Quorums | `quorum listextended` at `c`, `quorum info` for each new quorum | `update_quorum_info/` | Platform state: the validator sets and the chain lock and instant lock quorums | Other steps read Core at `c` or below too. The paths in the first two rows are @@ -409,6 +409,17 @@ under `packages/rs-drive-abci/src/execution/platform_events/withdrawals/`: | 8 | The statuses of the asset unlocks Platform broadcast (the Core transactions that pay out withdrawals) | `getassetunlockstatuses` at `c`, in blocks that move `c` | `update_broadcasted_withdrawal_statuses/` | The withdrawal documents | | 11 | Chain asset lock proofs (Dash locked on Core to fund an identity, proven by the chain-locked height that includes it) | The asset lock transaction, which must be mined at or below the proof's core height, itself no higher than Platform's last committed core height | `packages/rs-drive-abci/src/execution/validation/state_transition/common/asset_lock/` | What the state transition writes | +The first block of protocol version 15 reads Core once more, before step 1, in +its protocol change events (`transition_to_version_15`, under +`packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/`): Core's +full masternode list, `protx listdiff` from the first block to the previous +block's `c`, to reconcile the operator payouts Platform holds, and the operator +identities' TRANSFER keys, with Core's (`reconcile_operator_payouts_with_core/` +under `core_based_updates/`). That height is chain locked and an earlier block +already read Core there, so every validator's Core holds it. Unlike the reads +above, it runs whether or not the block moves `c`: a node whose Core cannot +answer cannot execute that one block. + ### When Core fails Core RPC calls retry for about 11 seconds on connection errors and on the diff --git a/book/src/architecture/core-fork-activations.md b/book/src/architecture/core-fork-activations.md index a54a34308f0..36031270549 100644 --- a/book/src/architecture/core-fork-activations.md +++ b/book/src/architecture/core-fork-activations.md @@ -151,10 +151,15 @@ and `perform_events_on_first_block_of_protocol_change`, run before `update_core_info`, so they cannot rely on the record for a fork that the same upgrade starts tracking. -Closing the window would mean asking Core in the first block of the new protocol -version whatever its `c`. That would put Core on the upgrade block's critical -path: a node whose Core failed could not execute the upgrade block, and the -upgrade would wait for every Core that was down. +Closing the window would mean asking Core about the fork in the first block of +the new protocol version whatever its `c`. That would put Core on the upgrade +block's critical path: a node whose Core failed could not execute the upgrade +block, and the upgrade would wait for every Core that was down. The upgrade to +protocol version 15 pays that cost once for another reason: it reconciles the +operator payouts Platform holds with Core's full masternode list (see +[Syncing with Dash Core](component-pipeline.md#syncing-with-dash-core)), which +nothing short of that list can do. The fork record does not need it: its window +is short, and every consumer must be safe in it anyway. ## V24's signal @@ -199,11 +204,13 @@ it failed, which is safe only for version 1. Pooling runs after records V24 is already version 2. In the window before the record, withdrawals stay at version 1, which Core always accepts. Pooling never asks Core about V24. -One piece of code still asks Core about V24 directly: the masternode list -update's `drop_unchanged_operator_keys`, which keeps a masternode's operator key -when Core re-encodes it after V24. It runs under every protocol version, and -from protocol version 15 it runs after the record is updated, in blocks that -move `c`, so it gets the same answer as the record. +The masternode list update keeps a masternode's operator key when Core +re-encodes it after V24. From protocol version 15, `update_masternode_list` 2 +reads V24 from the record. It runs inside `update_core_info` v1, after the +record step at the same `c` and only in blocks that move `c`, so `None` there +means Core reported V24 inactive at this very `c`. `update_masternode_list` 0 +and 1, up to protocol version 14, still ask Core directly +(`drop_unchanged_operator_keys`), as the blocks they executed did. ## Adding a fork diff --git a/packages/dashmate/configs/defaults/getBaseConfigFactory.js b/packages/dashmate/configs/defaults/getBaseConfigFactory.js index 5ff509643ae..ebea9c7c7d7 100644 --- a/packages/dashmate/configs/defaults/getBaseConfigFactory.js +++ b/packages/dashmate/configs/defaults/getBaseConfigFactory.js @@ -94,7 +94,7 @@ export default function getBaseConfigFactory() { 'getbestchainlock', 'getblockchaininfo', 'getrawtransaction', 'submitchainlock', 'verifychainlock', 'protxlistdiff', 'quorumlistextended', 'quoruminfo', 'getassetunlockstatuses', 'sendrawtransaction', 'mnsyncstatus', 'getblockheader', 'getblockhash', - 'getspecialtxes', 'getdeploymentinfo', + 'getspecialtxes', 'getdeploymentinfo', 'protxinfo', ], lowPriority: false, }, diff --git a/packages/dashmate/configs/getConfigFileMigrationsFactory.js b/packages/dashmate/configs/getConfigFileMigrationsFactory.js index 04901589b4a..14debd15172 100644 --- a/packages/dashmate/configs/getConfigFileMigrationsFactory.js +++ b/packages/dashmate/configs/getConfigFileMigrationsFactory.js @@ -1868,6 +1868,22 @@ export default function getConfigFileMigrationsFactory(homeDir, defaultConfigs) return configFile; }, + '5.1.0-beta.1': (configFile) => { + Object.entries(configFile.configs) + .forEach(([, options]) => { + // From protocol version 15 Drive reads, with protx info, the operator payout of a + // masternode whose operator key changed. Core refuses the method to the consensus + // user until its whitelist names it, and loads the whitelist only when Core itself + // restarts. Added to the operator's list rather than replacing it; a user without a + // whitelist may already call every method. + const whitelist = options.core?.rpc?.users?.drive_consensus?.whitelist; + if (Array.isArray(whitelist) && !whitelist.includes('protxinfo')) { + whitelist.push('protxinfo'); + } + }); + + return configFile; + }, }; } diff --git a/packages/dashmate/test/unit/config/configFile/migrateConfigFileFactory.spec.js b/packages/dashmate/test/unit/config/configFile/migrateConfigFileFactory.spec.js index 1ebc8da3a7a..b7ec53c32d7 100644 --- a/packages/dashmate/test/unit/config/configFile/migrateConfigFileFactory.spec.js +++ b/packages/dashmate/test/unit/config/configFile/migrateConfigFileFactory.spec.js @@ -51,6 +51,10 @@ describe('migrateConfigFileFactory', () => { whitelist.splice(whitelist.indexOf(method), 1); }; + if (semver.lt(release, '5.1.0-beta.1')) { + removeFromWhitelist('protxinfo'); + } + if (semver.lt(release, '5.0.0-rc.2')) { removeFromWhitelist('getdeploymentinfo'); } @@ -644,6 +648,40 @@ describe('migrateConfigFileFactory', () => { } }); + it('should let Drive call protx info on Core from a 5.0.3 config', async () => { + // From protocol version 15 Drive reads, with protx info, the operator payout of a + // masternode whose operator key changed, and Core answers a method missing from the + // consensus user's whitelist with HTTP 403. + const fromVersion = '5.0.3'; + const { version } = JSON.parse(fs.readFileSync(path.join(PACKAGE_ROOT_DIR, 'package.json'), 'utf8')); + + const configFileData = createConfigFile().toObject(); + configFileData.configFormatVersion = fromVersion; + for (const options of Object.values(configFileData.configs)) { + const { whitelist } = options.core.rpc.users.drive_consensus; + expect(whitelist).to.include('protxinfo'); + whitelist.splice(whitelist.indexOf('protxinfo'), 1); + } + configFileData.configs.testnet.core.rpc.users.drive_consensus.whitelist.push('getblockcount'); + configFileData.configs.mainnet.core.rpc.users.drive_consensus.whitelist = null; + + const migrated = migrateConfigFile(configFileData, fromVersion, version); + + expect(migrated.configs.base.core.rpc.users.drive_consensus.whitelist) + .to.include('protxinfo'); + expect(migrated.configs.local.core.rpc.users.drive_consensus.whitelist) + .to.include('protxinfo'); + // A method the operator allowed stays allowed. + expect(migrated.configs.testnet.core.rpc.users.drive_consensus.whitelist) + .to.include.members(['protxinfo', 'getblockcount']); + // A user without a whitelist may already call every method. + expect(migrated.configs.mainnet.core.rpc.users.drive_consensus.whitelist).to.equal(null); + + for (const [name, options] of Object.entries(migrated.configs)) { + expect(() => new Config(name, options), `migrated ${name} config does not load`).to.not.throw(); + } + }); + it('should keep an operator image that predates the 4.0.0 re-pin', async () => { // Every config older than 4.0.0 crosses the unconditional re-pin in that // migration, so it is the first place operator intent can be respected. It diff --git a/packages/rs-drive-abci/src/execution/engine/run_block_proposal/mod.rs b/packages/rs-drive-abci/src/execution/engine/run_block_proposal/mod.rs index 2f62b61c6a2..793dffcf890 100644 --- a/packages/rs-drive-abci/src/execution/engine/run_block_proposal/mod.rs +++ b/packages/rs-drive-abci/src/execution/engine/run_block_proposal/mod.rs @@ -158,6 +158,7 @@ Your software version: {}, latest supported protocol version: {}."#, // This is for events like adding stuff to the root tree, or making structural changes/fixes self.perform_events_on_first_block_of_protocol_change( platform_state, + &mut block_platform_state, &block_info, transaction, old_protocol_version, diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/mod.rs index f0a8e18930a..494454d9063 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/mod.rs @@ -1,3 +1,4 @@ +mod reconcile_operator_payouts_with_core; mod update_core_info; mod update_masternode_identities; mod update_masternode_list; diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/reconcile_operator_payouts_with_core/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/reconcile_operator_payouts_with_core/mod.rs new file mode 100644 index 00000000000..3255fad4321 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/reconcile_operator_payouts_with_core/mod.rs @@ -0,0 +1,531 @@ +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::platform_types::platform_state::PlatformStateV0Methods; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::dashcore::ProTxHash; +use dpp::dashcore_rpc::dashcore_rpc_json::{DMNStateDiff, MasternodeListDiff}; +use dpp::version::PlatformVersion; +use drive::grovedb::Transaction; +use std::collections::BTreeMap; + +/// How many masternodes of each side a membership mismatch names. +const MISMATCHES_REPORTED: usize = 10; + +impl Platform +where + C: CoreRPCLike, +{ + /// Makes the stored operator payouts, and the TRANSFER keys of each masternode's current + /// operator identity, match Dash Core's masternode list at `core_height`. Runs once, in the + /// first block of protocol version 15. + /// + /// Before protocol version 15 Platform took an operator payout only from a masternode's + /// full state, when it first listed the masternode, so a payout set or changed in Core later + /// never reached Platform, and a new operator inherited the previous operator's payout key. + /// From version 15 payouts follow Core's diffs; this fixes what earlier versions left. A + /// stored payout that differs from Core's is replaced. Every masternode's current operator + /// identity goes through `update_operator_identity` with Core's payout, which reconciles all + /// of the identity's TRANSFER keys with it and changes nothing where they already match: + /// the stored payout does not tell which identities need it, as earlier versions matched + /// platform node id keys by their data alone, so an evonode whose node id once had its + /// payout's bytes can have its payout key disabled while the stored payout and Core's agree. + /// An identity a masternode already left is not reconciled: a switch back to it leaves its + /// TRANSFER keys, and its next payout change reconciles them. + /// + /// `block_platform_state` must hold Platform's copy of Core's list at `core_height`, a chain + /// locked height every node agrees on, so every node reads the same list from its Core. A + /// Core error, or a Core list whose masternodes are not the ones Platform lists, fails the + /// block: both mean that this node cannot tell what the others reconcile. + /// + /// Part of protocol version 15's upgrade block (`transition_to_version_15`), its only caller: + /// once that version ships, a change in what this does needs a new step in the upgrade + /// ladder, not an edit here. It relies on an `update_operator_identity` generation that + /// reconciles every TRANSFER key with the payout, 1 or later. + pub(in crate::execution::platform_events) fn reconcile_operator_payouts_with_core( + &self, + block_platform_state: &mut PlatformState, + core_height: u32, + block_info: &BlockInfo, + transaction: &Transaction, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + // From the first block, so every masternode at `core_height` is in `added_mns`. + let core_list = self + .core_rpc + .get_protx_diff_with_masternodes(None, core_height) + .inspect_err(|error| { + tracing::error!( + core_height, + ?error, + "could not read Dash Core's masternode list to reconcile operator payouts" + ) + })?; + if !core_list.removed_mns.is_empty() || !core_list.updated_mns.is_empty() { + return Err(ExecutionError::DashCoreBadResponseError(format!( + "the masternode list from the first block to core height {core_height} removes or updates masternodes" + )) + .into()); + } + let listed_by_core = core_list.added_mns.len(); + let core_payouts = core_list + .added_mns + .into_iter() + .map(|masternode| { + ( + masternode.pro_tx_hash, + masternode.state.operator_payout_address, + ) + }) + .collect::>>(); + if core_payouts.len() != listed_by_core { + return Err(ExecutionError::DashCoreBadResponseError(format!( + "the masternode list at core height {core_height} lists a masternode more than once" + )) + .into()); + } + + let stored_list = block_platform_state.full_masternode_list(); + if !stored_list.keys().eq(core_payouts.keys()) { + // The first few of each side are enough to find what diverged and why. + let not_in_core = stored_list + .keys() + .filter(|pro_tx_hash| !core_payouts.contains_key(*pro_tx_hash)) + .collect::>(); + let not_listed = core_payouts + .keys() + .filter(|pro_tx_hash| !stored_list.contains_key(*pro_tx_hash)) + .collect::>(); + let message = format!( + "Platform's masternode list at core height {core_height} is not Dash Core's: \ + {} listed masternodes are not in Core's list (first: {:?}) and {} of Core's \ + are not listed (first: {:?})", + not_in_core.len(), + ¬_in_core[..not_in_core.len().min(MISMATCHES_REPORTED)], + not_listed.len(), + ¬_listed[..not_listed.len().min(MISMATCHES_REPORTED)], + ); + tracing::error!(core_height, "{message}"); + return Err(ExecutionError::CorruptedCachedState(message).into()); + } + + // Both lists hold the same masternodes in the same order. + let corrections = stored_list + .values() + .zip(core_payouts.values()) + .filter(|(masternode, core_payout)| { + masternode.state.operator_payout_address != **core_payout + }) + .map(|(masternode, core_payout)| (masternode.clone(), *core_payout)) + .collect::>(); + + tracing::info!( + core_height, + masternodes = core_payouts.len(), + corrected = corrections.len(), + "reconciling the operator payouts of the masternode list with Dash Core" + ); + + for (mut masternode, core_payout) in corrections { + masternode.state.operator_payout_address = core_payout; + block_platform_state.insert_masternode(masternode); + } + let updated_mns = core_payouts + .into_iter() + .map(|(pro_tx_hash, core_payout)| (pro_tx_hash, operator_payout_diff(core_payout))) + .collect(); + + // The corrected list is the state the identity updates read: its operator keys name the + // current operator identities. + self.update_masternode_identities( + MasternodeListDiff { + base_height: core_height, + block_height: core_height, + added_mns: vec![], + removed_mns: vec![], + updated_mns, + }, + &BTreeMap::new(), + block_info, + Some(&*block_platform_state), + transaction, + platform_version, + ) + } +} + +/// A Core state diff that sets only the operator payout. +fn operator_payout_diff(operator_payout_address: Option<[u8; 20]>) -> DMNStateDiff { + DMNStateDiff { + service: None, + registered_height: None, + last_paid_height: None, + consecutive_payments: None, + pose_penalty: None, + pose_revived_height: None, + pose_ban_height: None, + revocation_reason: None, + owner_address: None, + voting_address: None, + payout_address: None, + payouts: None, + pub_key_operator: None, + operator_payout_address: Some(operator_payout_address), + platform_node_id: None, + #[allow(deprecated)] + legacy_platform_p2p_port: None, + #[allow(deprecated)] + legacy_platform_http_port: None, + addresses: None, + } +} + +#[cfg(test)] +mod tests { + use crate::config::PlatformConfig; + use crate::error::execution::ExecutionError; + use crate::error::Error; + use crate::platform_types::platform_state::{PlatformState, PlatformStateV0Methods}; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; + use dpp::block::block_info::BlockInfo; + use dpp::dashcore::hashes::Hash; + use dpp::dashcore::{ProTxHash, Txid}; + use dpp::dashcore_rpc::dashcore_rpc_json::{ + DMNState, MasternodeListDiff, MasternodeListItem, MasternodeType, + }; + use dpp::dashcore_rpc::Error as CoreRpcError; + use dpp::identifier::MasternodeIdentifiers; + use dpp::identity::accessors::IdentityGettersV0; + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dpp::identity::Purpose; + use dpp::prelude::Identifier; + use dpp::version::PlatformVersion; + use drive::grovedb::Transaction; + use std::collections::BTreeSet; + use std::net::SocketAddr; + use std::sync::{Arc, Mutex}; + + /// The Core height the masternodes are first listed at, and the one reconciled at. + const LISTED_AT: u32 = 1000; + const RECONCILED_AT: u32 = 1001; + + /// A Regular masternode named by `id` with an operator reward and `operator_payout`. + fn masternode(id: u8, operator_payout: Option) -> MasternodeListItem { + MasternodeListItem { + node_type: MasternodeType::Regular, + pro_tx_hash: ProTxHash::from_byte_array([id; 32]), + collateral_hash: Txid::from_byte_array([id ^ 0x40; 32]), + collateral_index: 0, + collateral_address: Some([id ^ 0x50; 20]), + operator_reward: 10.0, + state: DMNState { + service: SocketAddr::from(([192, 0, 2, id], 9999)), + registered_height: 900, + pose_revived_height: None, + pose_ban_height: None, + revocation_reason: 0, + owner_address: Some([id ^ 0x10; 20]), + voting_address: [id ^ 0x20; 20], + payout_address: Some([id ^ 0x30; 20]), + payouts: None, + pub_key_operator: vec![id; 48], + operator_payout_address: operator_payout.map(|payout| [payout; 20]), + platform_node_id: None, + #[allow(deprecated)] + legacy_platform_p2p_port: None, + #[allow(deprecated)] + legacy_platform_http_port: None, + addresses: None, + }, + } + } + + type CoreCalls = Arc, u32)>>>; + + /// A platform at the latest protocol version whose Core lists `listed` at `LISTED_AT` and + /// answers `core_list` for its list at `RECONCILED_AT`, once. Returns the masternode list + /// calls made to Core at `RECONCILED_AT`. + fn platform_with_core( + listed: Vec, + core_list: Result, + ) -> (TempPlatform, CoreCalls) { + let mut platform = TestPlatformBuilder::new() + .with_config(PlatformConfig::default()) + .build_with_mock_rpc() + .set_genesis_state(); + let listed_diff = MasternodeListDiff { + base_height: 1, + block_height: LISTED_AT, + added_mns: listed, + removed_mns: vec![], + updated_mns: vec![], + }; + let core_list = Mutex::new(Some(core_list)); + let calls = CoreCalls::default(); + let recorded_calls = Arc::clone(&calls); + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |base_block, block| { + if block == LISTED_AT { + return Ok(listed_diff.clone()); + } + recorded_calls + .lock() + .expect("expected the Core calls") + .push((base_block, block)); + core_list + .lock() + .expect("expected Core's list") + .take() + .expect("expected one masternode list call at the reconciled height") + }); + (platform, calls) + } + + /// Core's full list of `masternodes` at `RECONCILED_AT`. + fn full_list(masternodes: Vec) -> MasternodeListDiff { + MasternodeListDiff { + base_height: 1, + block_height: RECONCILED_AT, + added_mns: masternodes, + removed_mns: vec![], + updated_mns: vec![], + } + } + + /// Lists the masternodes at `LISTED_AT` as init chain does, creating their identities. + /// Returns the platform state, with nothing pending to store. + fn list_masternodes( + platform: &TempPlatform, + transaction: &Transaction, + ) -> PlatformState { + let mut state = platform.state.load().as_ref().clone(); + platform + .update_masternode_list( + None, + &mut state, + LISTED_AT, + true, + &BlockInfo { + height: 1, + core_height: LISTED_AT, + ..Default::default() + }, + transaction, + PlatformVersion::latest(), + ) + .expect("expected to list the masternodes"); + state.mark_saved(); + state + } + + fn reconcile( + platform: &TempPlatform, + state: &mut PlatformState, + transaction: &Transaction, + ) -> Result<(), Error> { + platform.reconcile_operator_payouts_with_core( + state, + RECONCILED_AT, + &BlockInfo { + height: 2, + core_height: RECONCILED_AT, + ..Default::default() + }, + transaction, + PlatformVersion::latest(), + ) + } + + /// The (data, disabled) of every TRANSFER key of `masternode`'s operator identity. + fn transfer_keys( + platform: &TempPlatform, + transaction: &Transaction, + masternode: &MasternodeListItem, + ) -> Vec<(Vec, bool)> { + platform + .drive + .fetch_full_identity( + Identifier::create_operator_identifier( + masternode.pro_tx_hash.as_byte_array(), + &masternode.state.pub_key_operator, + ) + .to_buffer(), + Some(transaction), + PlatformVersion::latest(), + ) + .expect("expected to fetch the operator identity") + .expect("expected the operator identity") + .public_keys() + .values() + .filter(|key| key.purpose() == Purpose::TRANSFER) + .map(|key| (key.data().to_vec(), key.is_disabled())) + .collect() + } + + /// Stored payouts that went stale against Core's list (changed, cleared and first set after + /// Platform listed the masternode) are corrected in the stored list and on the current + /// operator identities, with one list call at the reconciled height. A masternode whose + /// payout matches is left as it is. + #[test] + fn should_fix_stale_operator_payouts_and_the_current_identities_keys() { + let changed = masternode(0x61, Some(0x61)); + let cleared = masternode(0x62, Some(0x62)); + // An evonode, which the evonode list holds too. + let mut first_set = masternode(0x63, None); + first_set.node_type = MasternodeType::Evo; + first_set.state.platform_node_id = Some([0x73; 20]); + #[allow(deprecated)] + { + first_set.state.legacy_platform_p2p_port = Some(26656); + first_set.state.legacy_platform_http_port = Some(443); + } + let matching = masternode(0x64, Some(0x64)); + let in_core = |masternode: &MasternodeListItem, payout: Option| { + let mut masternode = masternode.clone(); + masternode.state.operator_payout_address = payout.map(|payout| [payout; 20]); + masternode + }; + let (platform, calls) = platform_with_core( + vec![ + changed.clone(), + cleared.clone(), + first_set.clone(), + matching.clone(), + ], + Ok(full_list(vec![ + in_core(&changed, Some(0x81)), + in_core(&cleared, None), + in_core(&first_set, Some(0x83)), + matching.clone(), + ])), + ); + let transaction = platform.drive.grove.start_transaction(); + let mut state = list_masternodes(&platform, &transaction); + + reconcile(&platform, &mut state, &transaction).expect("expected to reconcile"); + + assert_eq!(*calls.lock().unwrap(), vec![(None, RECONCILED_AT)]); + for (masternode, payout) in [ + (&changed, Some([0x81; 20])), + (&cleared, None), + (&first_set, Some([0x83; 20])), + (&matching, Some([0x64; 20])), + ] { + assert_eq!( + state.full_masternode_list()[&masternode.pro_tx_hash] + .state + .operator_payout_address, + payout, + "{}", + masternode.pro_tx_hash + ); + } + assert_eq!( + state.hpmn_masternode_list()[&first_set.pro_tx_hash] + .state + .operator_payout_address, + Some([0x83; 20]) + ); + assert_eq!( + state.masternode_changes.upserted, + BTreeSet::from([ + changed.pro_tx_hash, + cleared.pro_tx_hash, + first_set.pro_tx_hash + ]) + ); + assert_eq!( + transfer_keys(&platform, &transaction, &changed), + [(vec![0x61; 20], true), (vec![0x81; 20], false)] + ); + assert_eq!( + transfer_keys(&platform, &transaction, &cleared), + [(vec![0x62; 20], true)] + ); + assert_eq!( + transfer_keys(&platform, &transaction, &first_set), + [(vec![0x83; 20], false)] + ); + assert_eq!( + transfer_keys(&platform, &transaction, &matching), + [(vec![0x64; 20], false)] + ); + } + + /// Core listing a masternode Platform does not, or not listing one it does, means this node + /// cannot tell what the others reconcile: the block fails, and nothing is changed. + #[test] + fn should_fail_when_core_lists_other_masternodes() { + let listed = masternode(0x65, Some(0x65)); + let other = masternode(0x66, Some(0x86)); + for core_list in [ + vec![], + vec![listed.clone(), other.clone()], + vec![other.clone()], + ] { + let (platform, _) = platform_with_core(vec![listed.clone()], Ok(full_list(core_list))); + let transaction = platform.drive.grove.start_transaction(); + let mut state = list_masternodes(&platform, &transaction); + let before = state.clone(); + + assert!(matches!( + reconcile(&platform, &mut state, &transaction), + Err(Error::Execution(ExecutionError::CorruptedCachedState(_))) + )); + assert_eq!(state.full_masternode_list(), before.full_masternode_list()); + assert!(state.masternode_changes.upserted.is_empty()); + } + } + + /// A Core error fails the block instead of skipping the reconciliation. + #[test] + fn should_fail_when_core_cannot_list_its_masternodes() { + let listed = masternode(0x67, Some(0x67)); + let (platform, _) = platform_with_core( + vec![listed.clone()], + Err(CoreRpcError::UnexpectedStructure("refused".to_string())), + ); + let transaction = platform.drive.grove.start_transaction(); + let mut state = list_masternodes(&platform, &transaction); + + assert!(reconcile(&platform, &mut state, &transaction).is_err()); + assert_eq!( + state.full_masternode_list()[&listed.pro_tx_hash] + .state + .operator_payout_address, + Some([0x67; 20]) + ); + } + + /// A full list from the first block cannot remove or update masternodes, nor list one twice: + /// such an answer fails the block instead of being reconciled against. + #[test] + fn should_fail_when_core_answers_something_other_than_a_full_list() { + let listed = masternode(0x68, Some(0x68)); + let mut removes = full_list(vec![listed.clone()]); + removes.removed_mns = vec![listed.pro_tx_hash]; + let mut updates = full_list(vec![listed.clone()]); + updates.updated_mns = vec![( + listed.pro_tx_hash, + serde_json::from_str("{}").expect("an empty Core state diff"), + )]; + let twice = full_list(vec![listed.clone(), listed.clone()]); + for core_list in [removes, updates, twice] { + let (platform, _) = platform_with_core(vec![listed.clone()], Ok(core_list)); + let transaction = platform.drive.grove.start_transaction(); + let mut state = list_masternodes(&platform, &transaction); + + assert!(matches!( + reconcile(&platform, &mut state, &transaction), + Err(Error::Execution(ExecutionError::DashCoreBadResponseError( + _ + ))) + )); + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/mod.rs index 220660f7493..dea90202b33 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/mod.rs @@ -1,4 +1,5 @@ mod v0; +mod v1; use crate::error::execution::ExecutionError; use crate::error::Error; @@ -70,9 +71,19 @@ where drive_operations, platform_version, ), + 1 => self.update_operator_identity_v1( + masternode_pro_tx_hash, + pub_key_operator_change, + operator_payout_address_change, + platform_node_id_change, + platform_state, + transaction, + drive_operations, + platform_version, + ), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "update_operator_identity".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v0/mod.rs index d5727a9652d..40630e61aec 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v0/mod.rs @@ -414,7 +414,7 @@ mod tests { platform: &TempPlatform, rng: &mut StdRng, ) -> (ProTxHash, Identity, [u8; 20], Vec, [u8; 20]) { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); // Create a dummy ProTxHash let pro_tx_hash_bytes: [u8; 32] = rng.gen(); @@ -507,8 +507,9 @@ mod tests { #[test] fn test_update_operator_payout_address() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -587,8 +588,9 @@ mod tests { #[test] fn test_update_operator_change_back_to_previous_operator_address() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -726,8 +728,9 @@ mod tests { #[test] fn test_update_operator_platform_node_id() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -806,8 +809,9 @@ mod tests { #[test] fn test_update_operator_change_back_to_previous_node_id() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -947,8 +951,9 @@ mod tests { #[test] fn test_update_operator_public_key() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -1044,8 +1049,9 @@ mod tests { #[test] fn test_update_operator_identity_reuses_node_id_without_change() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); @@ -1185,8 +1191,9 @@ mod tests { #[test] fn test_update_operator_change_back_to_previous_public_key() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .build_with_mock_rpc() .set_genesis_state(); diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v1/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v1/mod.rs new file mode 100644 index 00000000000..69962c11a4a --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_identities/update_operator_identity/v1/mod.rs @@ -0,0 +1,874 @@ +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::platform_types::platform_state::PlatformStateV0Methods; +use crate::rpc::core::CoreRPCLike; + +use dpp::dashcore::ProTxHash; +use dpp::identifier::{Identifier, MasternodeIdentifiers}; +use dpp::identity::accessors::IdentityGettersV0; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; +use dpp::identity::Purpose::TRANSFER; +use dpp::identity::{Identity, IdentityPublicKey, KeyID, KeyType, Purpose, SecurityLevel}; +use dpp::platform_value::BinaryData; +use dpp::version::PlatformVersion; +use drive::drive::identity::key::fetch::{ + IdentityKeysRequest, KeyIDIdentityPublicKeyPairBTreeMap, KeyRequestType, +}; +use drive::grovedb::Transaction; +use drive::util::batch::DriveOperation; +use drive::util::batch::DriveOperation::IdentityOperation; +use drive::util::batch::IdentityOperationType::{ + AddNewIdentity, AddNewKeysToMasternodeIdentity, DisableIdentityKeys, ReEnableIdentityKeys, +}; + +/// The key operations of one update of one operator identity. +#[derive(Default)] +struct KeyChanges { + to_disable: Vec, + to_re_enable: Vec, + to_add: Vec, +} + +/// Whether `key` is an operator identity's platform node id key. +fn is_platform_node_id_key(key: &IdentityPublicKey) -> bool { + key.purpose() == Purpose::SYSTEM && key.key_type() == KeyType::EDDSA_25519_HASH160 +} + +impl Platform +where + C: CoreRPCLike, +{ + /// Updates the operator identity like version 0, except for its TRANSFER keys. + /// + /// Version 0 was written for diffs that never carry an operator payout: it disables the key + /// of the stored payout, and gives a new identity an existing TRANSFER key or the stored + /// payout, which belong to an earlier operator. In this version the TRANSFER keys of an + /// identity follow the payout changes Core reports while it stays the masternode's operator + /// identity: given one, every TRANSFER key of the identity is reconciled with it + /// (`reconcile_operator_payout_keys_v1`), so applying the same payout twice changes nothing + /// the second time, and a cleared payout leaves no TRANSFER key enabled. + /// + /// A key change gives a new identity the payout Core reports with it; without one the new + /// identity gets no TRANSFER key, as Core empties the payout whenever the operator key + /// changes. A key change never touches the TRANSFER keys of an identity that already exists, + /// the one left behind or one the masternode switches back to, whatever the payout: the owner + /// alone authorizes a key change, so the payout reported with it need not be the returning + /// operator's own choice, and the operators of those identities must still be able to + /// withdraw what was sent to them. A key an earlier operator left on an identity the + /// masternode switches back to stays until that identity's next payout change. + /// + /// Platform node ids are handled as in version 0, except that each step touches only its own + /// kind of key (the BLS operator key, the platform node id key, the TRANSFER keys), so one key + /// is never disabled or re-enabled twice in one update. A key whose hash another key of the + /// identity has, which Core's payouts and node ids allow, is added as any other + /// (`add_new_keys_to_masternode_identity` 1). + #[allow(clippy::too_many_arguments)] + pub(super) fn update_operator_identity_v1( + &self, + masternode_pro_tx_hash: &ProTxHash, + pub_key_operator_change: Option<&Vec>, + operator_payout_address_change: Option>, + platform_node_id_change: Option<[u8; 20]>, + platform_state: &PlatformState, + transaction: &Transaction, + drive_operations: &mut Vec, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + if pub_key_operator_change.is_none() + && operator_payout_address_change.is_none() + && platform_node_id_change.is_none() + { + return Ok(()); + } + + let old_masternode = platform_state + .full_masternode_list() + .get(masternode_pro_tx_hash) + .ok_or_else(|| { + Error::Execution(ExecutionError::CorruptedCachedState(format!( + "expected masternode {} to be in state", + masternode_pro_tx_hash + ))) + })?; + + let old_operator_identifier = Self::get_operator_identifier_from_masternode_list_item( + old_masternode, + platform_version, + )?; + + let old_identity_keys = self.fetch_operator_identity_keys_v1( + old_operator_identifier, + transaction, + platform_version, + )?; + + let Some(pub_key_operator_change) = pub_key_operator_change else { + // Same identity. + let Some(mut last_key_id) = old_identity_keys.keys().max().copied() else { + return Err(ExecutionError::DriveMissingData(format!( + "expected the operator identity of masternode {} to be in state", + masternode_pro_tx_hash + )) + .into()); + }; + let mut changes = KeyChanges::default(); + + if let Some(platform_node_id_change) = platform_node_id_change { + let mut old_operator_node_id_to_re_enable = None; + if let Some(old_platform_node_id) = old_masternode.state.platform_node_id.as_ref() { + for (key_id, key) in old_identity_keys + .iter() + .filter(|(_, key)| is_platform_node_id_key(key)) + { + if key.data().as_slice() == old_platform_node_id { + changes.to_disable.push(*key_id); + } else if key.data().as_slice() == platform_node_id_change { + old_operator_node_id_to_re_enable = Some(*key_id); + } + } + } + if let Some(key_id) = old_operator_node_id_to_re_enable { + changes.to_re_enable.push(key_id); + } else { + changes.to_add.push(Self::platform_node_id_key_v1( + Self::next_operator_key_id_v1(&mut last_key_id)?, + platform_node_id_change, + )); + } + } + + if let Some(operator_payout_address) = operator_payout_address_change { + Self::reconcile_operator_payout_keys_v1( + &old_identity_keys, + operator_payout_address, + &mut last_key_id, + &mut changes, + )?; + } + + Self::push_key_changes_v1(old_operator_identifier, changes, drive_operations); + return Ok(()); + }; + + // Identity switch: the masternode moves to the identity of its new operator key, which + // may already exist when an earlier key comes back. + let operator_identifier = Identifier::create_operator_identifier( + masternode_pro_tx_hash.as_byte_array(), + pub_key_operator_change, + ); + + // The identity left behind loses its operator and platform node id keys, and keeps its + // TRANSFER keys. + let old_operator_identity_key_ids_to_disable: Vec = old_identity_keys + .iter() + .filter(|(_, key)| { + (key.key_type() == KeyType::BLS12_381 + && key.data().as_slice() == old_masternode.state.pub_key_operator) + || (is_platform_node_id_key(key) + && old_masternode + .state + .platform_node_id + .is_some_and(|node_id| key.data().as_slice() == node_id)) + }) + .map(|(key_id, _)| *key_id) + .collect(); + + if !old_operator_identity_key_ids_to_disable.is_empty() { + drive_operations.push(IdentityOperation(DisableIdentityKeys { + identity_id: old_operator_identifier.to_buffer(), + keys_ids: old_operator_identity_key_ids_to_disable, + })); + } + + let new_payout_address = operator_payout_address_change.flatten(); + let new_platform_node_id = + platform_node_id_change.or(old_masternode.state.platform_node_id); + + let identity_keys = self.fetch_operator_identity_keys_v1( + operator_identifier, + transaction, + platform_version, + )?; + + let Some(mut last_key_id) = identity_keys.keys().max().copied() else { + let mut identity = + Identity::create_basic_identity(operator_identifier, platform_version)?; + identity.add_public_keys(Self::get_operator_identity_keys( + pub_key_operator_change.clone(), + new_payout_address, + new_platform_node_id, + platform_version, + )?); + drive_operations.push(IdentityOperation(AddNewIdentity { + identity, + is_masternode_identity: true, + })); + return Ok(()); + }; + + // An existing identity is one an earlier operator with this key left. A key change never + // touches its TRANSFER keys, whatever the payout: the owner alone authorizes it, so the + // payout need not be the returning operator's own choice, and that operator must still be + // able to withdraw what was sent to it. Its next payout change reconciles them. + let mut changes = KeyChanges::default(); + + if let Some(new_platform_node_id) = new_platform_node_id { + if let Some((key_id, found_old_key)) = identity_keys.iter().find(|(_, key)| { + is_platform_node_id_key(key) && key.data().as_slice() == new_platform_node_id + }) { + if found_old_key.is_disabled() { + changes.to_re_enable.push(*key_id); + } + } else { + changes.to_add.push(Self::platform_node_id_key_v1( + Self::next_operator_key_id_v1(&mut last_key_id)?, + new_platform_node_id, + )); + } + } + + Self::push_key_changes_v1(operator_identifier, changes, drive_operations); + Ok(()) + } + + /// Queues the changes that leave `operator_payout_address` the only enabled TRANSFER key + /// among `keys`, or no TRANSFER key enabled when it is `None`: a key holding the payout is + /// kept, an enabled one before a disabled one and lower ids first, and re-enabled if it is + /// disabled; otherwise a key is added, with the id after `last_key_id`, which it advances. + /// Every other enabled TRANSFER key is disabled. + fn reconcile_operator_payout_keys_v1( + keys: &KeyIDIdentityPublicKeyPairBTreeMap, + operator_payout_address: Option<[u8; 20]>, + last_key_id: &mut KeyID, + changes: &mut KeyChanges, + ) -> Result<(), Error> { + let reusable_key = operator_payout_address.and_then(|address| { + keys.iter() + .filter(|(_, key)| { + key.purpose() == Purpose::TRANSFER + && key.key_type() == KeyType::ECDSA_HASH160 + && key.data().as_slice() == address + }) + .min_by_key(|(key_id, key)| (key.is_disabled(), **key_id)) + }); + let reused_key_id = reusable_key.map(|(key_id, _)| *key_id); + changes.to_disable.extend( + keys.iter() + .filter(|(key_id, key)| { + key.purpose() == Purpose::TRANSFER + && !key.is_disabled() + && reused_key_id != Some(**key_id) + }) + .map(|(key_id, _)| *key_id), + ); + match (reusable_key, operator_payout_address) { + (Some((key_id, key)), _) => { + if key.is_disabled() { + changes.to_re_enable.push(*key_id); + } + } + (None, Some(address)) => { + changes.to_add.push( + IdentityPublicKeyV0 { + id: Self::next_operator_key_id_v1(last_key_id)?, + key_type: KeyType::ECDSA_HASH160, + purpose: TRANSFER, + security_level: SecurityLevel::CRITICAL, + read_only: true, + data: BinaryData::new(address.to_vec()), + disabled_at: None, + contract_bounds: None, + } + .into(), + ); + } + (None, None) => {} + } + Ok(()) + } + + fn fetch_operator_identity_keys_v1( + &self, + identity_id: Identifier, + transaction: &Transaction, + platform_version: &PlatformVersion, + ) -> Result { + Ok(self + .drive + .fetch_identity_keys::( + IdentityKeysRequest { + identity_id: identity_id.to_buffer(), + request_type: KeyRequestType::AllKeys, + limit: None, + offset: None, + }, + Some(transaction), + platform_version, + )?) + } + + /// The id for a key added after `last_key_id`, which becomes it. Only an added key takes an + /// id, so an identity at the last id can still have keys reused, disabled or re-enabled. + fn next_operator_key_id_v1(last_key_id: &mut KeyID) -> Result { + *last_key_id = last_key_id.checked_add(1).ok_or_else(|| { + Error::from(ExecutionError::Overflow( + "masternode operator identity key id exhausted", + )) + })?; + Ok(*last_key_id) + } + + fn platform_node_id_key_v1(key_id: KeyID, platform_node_id: [u8; 20]) -> IdentityPublicKey { + IdentityPublicKeyV0 { + id: key_id, + key_type: KeyType::EDDSA_25519_HASH160, + purpose: Purpose::SYSTEM, + security_level: SecurityLevel::CRITICAL, + read_only: true, + data: BinaryData::new(platform_node_id.to_vec()), + disabled_at: None, + contract_bounds: None, + } + .into() + } + + /// Queues `changes` on `identity_id`: disables first, then re-enables, then additions, as + /// version 0 orders them. + fn push_key_changes_v1( + identity_id: Identifier, + changes: KeyChanges, + drive_operations: &mut Vec, + ) { + let KeyChanges { + to_disable, + to_re_enable, + to_add, + } = changes; + if !to_disable.is_empty() { + drive_operations.push(IdentityOperation(DisableIdentityKeys { + identity_id: identity_id.to_buffer(), + keys_ids: to_disable, + })); + } + if !to_re_enable.is_empty() { + drive_operations.push(IdentityOperation(ReEnableIdentityKeys { + identity_id: identity_id.to_buffer(), + keys_ids: to_re_enable, + })); + } + if !to_add.is_empty() { + drive_operations.push(IdentityOperation(AddNewKeysToMasternodeIdentity { + identity_id: identity_id.to_buffer(), + non_unique_keys_to_add: to_add, + })); + } + } +} + +#[cfg(test)] +mod tests { + use crate::error::execution::ExecutionError; + use crate::error::Error; + use crate::platform_types::platform_state::{PlatformState, PlatformStateV0Methods}; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; + use dpp::block::block_info::BlockInfo; + use dpp::dashcore::hashes::Hash; + use dpp::dashcore::{ProTxHash, Txid}; + use dpp::dashcore_rpc::dashcore_rpc_json::{DMNState, MasternodeListItem, MasternodeType}; + use dpp::identifier::MasternodeIdentifiers; + use dpp::identity::accessors::IdentityGettersV0; + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; + use dpp::identity::{IdentityV0, KeyID, KeyType, Purpose, SecurityLevel}; + use dpp::platform_value::BinaryData; + use dpp::prelude::{Identifier, Identity, IdentityPublicKey}; + use dpp::version::PlatformVersion; + use std::collections::BTreeMap; + use std::net::SocketAddr; + + const PRO_TX_HASH: [u8; 32] = [0x41; 32]; + const PUB_KEY_OPERATOR: [u8; 48] = [0x42; 48]; + const NODE_ID: [u8; 20] = [0x43; 20]; + + /// A platform listing one masternode whose stored operator payout is `stored_payout`, and + /// whose operator identity holds the operator key, the platform node id key and a TRANSFER + /// key for each `(data, disabled)` of `transfer_keys`, with ids from 2 on. With + /// `create_identity` false the identity is not created. + fn platform_with_operator( + stored_payout: Option<[u8; 20]>, + transfer_keys: &[([u8; 20], bool)], + create_identity: bool, + ) -> (TempPlatform, PlatformState) { + let platform = TestPlatformBuilder::new() + .build_with_mock_rpc() + .set_genesis_state(); + if create_identity { + add_operator_identity(&platform, &PUB_KEY_OPERATOR, transfer_keys); + } + let mut platform_state = platform.state.load().as_ref().clone(); + platform_state.full_masternode_list_mut().insert( + ProTxHash::from_byte_array(PRO_TX_HASH), + MasternodeListItem { + node_type: MasternodeType::Regular, + pro_tx_hash: ProTxHash::from_byte_array(PRO_TX_HASH), + collateral_hash: Txid::from_byte_array([0x44; 32]), + collateral_index: 0, + collateral_address: Some([0x45; 20]), + operator_reward: 10.0, + state: DMNState { + service: SocketAddr::from(([192, 0, 2, 1], 9999)), + registered_height: 0, + pose_revived_height: None, + pose_ban_height: None, + revocation_reason: 0, + owner_address: Some([0x46; 20]), + voting_address: [0x47; 20], + payout_address: Some([0x48; 20]), + payouts: None, + pub_key_operator: PUB_KEY_OPERATOR.to_vec(), + operator_payout_address: stored_payout, + platform_node_id: Some(NODE_ID), + #[allow(deprecated)] + legacy_platform_p2p_port: None, + #[allow(deprecated)] + legacy_platform_http_port: None, + addresses: None, + }, + }, + ); + (platform, platform_state) + } + + /// Adds the operator identity of the masternode for operator key `pub_key_operator`, with + /// that key, the platform node id key and a TRANSFER key for each `(data, disabled)` of + /// `transfer_keys`, with ids from 2 on. + fn add_operator_identity( + platform: &TempPlatform, + pub_key_operator: &[u8], + transfer_keys: &[([u8; 20], bool)], + ) { + let key = |id: KeyID, key_type, purpose, data: &[u8], disabled: bool| { + IdentityPublicKeyV0 { + id, + key_type, + purpose, + security_level: SecurityLevel::CRITICAL, + read_only: true, + data: BinaryData::new(data.to_vec()), + disabled_at: disabled.then_some(1), + contract_bounds: None, + } + .into() + }; + let mut public_keys = BTreeMap::from([ + ( + 0, + key( + 0, + KeyType::BLS12_381, + Purpose::SYSTEM, + pub_key_operator, + false, + ), + ), + ( + 1, + key( + 1, + KeyType::EDDSA_25519_HASH160, + Purpose::SYSTEM, + &NODE_ID, + false, + ), + ), + ]); + for (key_id, (data, disabled)) in (2..).zip(transfer_keys) { + public_keys.insert( + key_id, + key( + key_id, + KeyType::ECDSA_HASH160, + Purpose::TRANSFER, + data, + *disabled, + ), + ); + } + let identity: Identity = IdentityV0 { + id: Identifier::create_operator_identifier(&PRO_TX_HASH, pub_key_operator), + public_keys, + balance: 0, + revision: 0, + } + .into(); + platform + .drive + .add_new_identity( + identity, + true, + &BlockInfo::default(), + true, + None, + PlatformVersion::latest(), + ) + .expect("expected to add the operator identity"); + } + + fn operator_identifier() -> Identifier { + Identifier::create_operator_identifier(&PRO_TX_HASH, &PUB_KEY_OPERATOR) + } + + /// Runs `update_operator_identity_v1` for the masternode with `pub_key_operator`, `payout` + /// and `platform_node_id` and applies what it queued. Returns how many operations it queued. + fn update( + platform: &TempPlatform, + platform_state: &PlatformState, + pub_key_operator: Option<&Vec>, + payout: Option>, + platform_node_id: Option<[u8; 20]>, + ) -> Result { + let platform_version = PlatformVersion::latest(); + let transaction = platform.drive.grove.start_transaction(); + let mut drive_operations = vec![]; + platform.update_operator_identity_v1( + &ProTxHash::from_byte_array(PRO_TX_HASH), + pub_key_operator, + payout, + platform_node_id, + platform_state, + &transaction, + &mut drive_operations, + platform_version, + )?; + let queued = drive_operations.len(); + platform + .drive + .apply_drive_operations( + drive_operations, + true, + &BlockInfo::default(), + Some(&transaction), + platform_version, + ) + .expect("expected to apply the key changes"); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected to commit"); + Ok(queued) + } + + /// The (id, purpose, data, disabled) of every key of the operator identity for operator key + /// `pub_key_operator`; `None` without that identity. + fn keys_of( + platform: &TempPlatform, + pub_key_operator: &[u8], + ) -> Option, bool)>> { + platform + .drive + .fetch_full_identity( + Identifier::create_operator_identifier(&PRO_TX_HASH, pub_key_operator).to_buffer(), + None, + PlatformVersion::latest(), + ) + .expect("expected to fetch the operator identity") + .map(|identity| { + identity + .public_keys() + .iter() + .map(|(id, key)| (*id, key.purpose(), key.data().to_vec(), key.is_disabled())) + .collect() + }) + } + + /// The (purpose, data, disabled) of every key of the operator identity, by key id. + fn keys(platform: &TempPlatform) -> Vec<(Purpose, Vec, bool)> { + platform + .drive + .fetch_full_identity( + operator_identifier().to_buffer(), + None, + PlatformVersion::latest(), + ) + .expect("expected to fetch the operator identity") + .expect("expected the operator identity") + .public_keys() + .values() + .map(|key| (key.purpose(), key.data().to_vec(), key.is_disabled())) + .collect() + } + + /// The payout's key is the only TRANSFER key left enabled, whatever the stored payout says: + /// a disabled key holding it is re-enabled, and every other enabled TRANSFER key disabled. + /// The second identical update queues nothing. + #[test] + fn should_leave_only_the_payout_key_enabled_and_change_nothing_the_second_time() { + let (platform, platform_state) = platform_with_operator( + Some([0x51; 20]), + &[([0x51; 20], false), ([0x52; 20], false), ([0x53; 20], true)], + true, + ); + assert!( + update( + &platform, + &platform_state, + None, + Some(Some([0x53; 20])), + None + ) + .unwrap() + > 0 + ); + assert_eq!( + keys(&platform)[2..], + [ + (Purpose::TRANSFER, vec![0x51; 20], true), + (Purpose::TRANSFER, vec![0x52; 20], true), + (Purpose::TRANSFER, vec![0x53; 20], false), + ] + ); + assert_eq!( + update( + &platform, + &platform_state, + None, + Some(Some([0x53; 20])), + None + ) + .unwrap(), + 0 + ); + } + + /// A cleared payout leaves no TRANSFER key enabled, and adds none. + #[test] + fn should_disable_every_transfer_key_when_the_payout_is_cleared() { + let (platform, platform_state) = platform_with_operator( + Some([0x51; 20]), + &[([0x51; 20], false), ([0x52; 20], false)], + true, + ); + update(&platform, &platform_state, None, Some(None), None).unwrap(); + assert_eq!( + keys(&platform)[2..], + [ + (Purpose::TRANSFER, vec![0x51; 20], true), + (Purpose::TRANSFER, vec![0x52; 20], true), + ] + ); + assert_eq!( + update(&platform, &platform_state, None, Some(None), None).unwrap(), + 0 + ); + } + + /// An update without a payout, here a platform node id change, leaves the TRANSFER keys as + /// they are, and handles the node id as version 0 does. + #[test] + fn should_leave_the_transfer_keys_of_an_update_without_a_payout() { + let (platform, platform_state) = platform_with_operator( + Some([0x51; 20]), + &[([0x51; 20], false), ([0x52; 20], false)], + true, + ); + update(&platform, &platform_state, None, None, Some([0x54; 20])).unwrap(); + assert_eq!( + keys(&platform), + [ + (Purpose::SYSTEM, PUB_KEY_OPERATOR.to_vec(), false), + (Purpose::SYSTEM, NODE_ID.to_vec(), true), + (Purpose::TRANSFER, vec![0x51; 20], false), + (Purpose::TRANSFER, vec![0x52; 20], false), + (Purpose::SYSTEM, vec![0x54; 20], false), + ] + ); + } + + /// Every listed masternode has an operator identity, so a missing one is corrupted state: + /// the update fails rather than add keys to an identity that does not exist. + #[test] + fn should_fail_when_the_operator_identity_is_missing() { + let (platform, platform_state) = platform_with_operator(Some([0x51; 20]), &[], false); + assert!(matches!( + update( + &platform, + &platform_state, + None, + Some(Some([0x52; 20])), + None + ), + Err(Error::Execution(ExecutionError::DriveMissingData(_))) + )); + } + + /// A platform node id and a payout that each need a new key get two keys with different ids. + #[test] + fn should_give_distinct_ids_to_the_keys_one_update_adds() { + let (platform, platform_state) = + platform_with_operator(Some([0x51; 20]), &[([0x51; 20], false)], true); + update( + &platform, + &platform_state, + None, + Some(Some([0x52; 20])), + Some([0x54; 20]), + ) + .unwrap(); + assert_eq!( + keys_of(&platform, &PUB_KEY_OPERATOR).expect("expected the operator identity")[1..], + [ + (1, Purpose::SYSTEM, NODE_ID.to_vec(), true), + (2, Purpose::TRANSFER, vec![0x51; 20], true), + (3, Purpose::SYSTEM, vec![0x54; 20], false), + (4, Purpose::TRANSFER, vec![0x52; 20], false), + ] + ); + } + + /// Core empties the operator payout whenever the operator key changes, so a key change + /// without a payout creates the new operator identity without a TRANSFER key, never with the + /// stored payout, which is the previous operator's. The identity left behind keeps it. + #[test] + fn should_create_a_new_operator_identity_without_the_previous_operators_payout_key() { + let (platform, platform_state) = + platform_with_operator(Some([0x51; 20]), &[([0x51; 20], false)], true); + let new_key = vec![0x62; 48]; + update(&platform, &platform_state, Some(&new_key), None, None).unwrap(); + let new_identity = keys_of(&platform, &new_key).expect("expected the new identity"); + assert!( + new_identity + .iter() + .all(|(_, purpose, ..)| *purpose != Purpose::TRANSFER), + "{new_identity:?}" + ); + assert_eq!( + keys(&platform), + [ + (Purpose::SYSTEM, PUB_KEY_OPERATOR.to_vec(), true), + (Purpose::SYSTEM, NODE_ID.to_vec(), true), + (Purpose::TRANSFER, vec![0x51; 20], false), + ] + ); + } + + /// A key change to an operator identity that already exists, an earlier operator's, leaves + /// its TRANSFER keys as they are whatever the payout: none is disabled, re-enabled or added, + /// not even for a payout one of them holds. The owner alone authorizes a key change, so the + /// payout need not be the returning operator's own choice, and the earlier operator can still + /// withdraw what was sent to it. The platform node id key is added under its own id. + #[test] + fn should_leave_the_transfer_keys_of_an_existing_identity_a_key_change_moves_to() { + for payout in [ + Some(Some([0x56; 20])), + Some(Some([0x58; 20])), + Some(None), + None, + ] { + let (platform, platform_state) = + platform_with_operator(Some([0x51; 20]), &[([0x51; 20], false)], true); + let earlier_key = vec![0x63; 48]; + add_operator_identity( + &platform, + &earlier_key, + &[([0x55; 20], false), ([0x56; 20], true)], + ); + update( + &platform, + &platform_state, + Some(&earlier_key), + payout, + Some([0x57; 20]), + ) + .unwrap(); + let identity = keys_of(&platform, &earlier_key).expect("expected the identity"); + assert_eq!( + identity + .iter() + .filter(|(_, purpose, ..)| *purpose == Purpose::TRANSFER) + .cloned() + .collect::>(), + [ + (2, Purpose::TRANSFER, vec![0x55; 20], false), + (3, Purpose::TRANSFER, vec![0x56; 20], true), + ], + "payout {payout:?}" + ); + let node_id_keys = identity + .iter() + .filter(|(_, _, data, _)| *data == vec![0x57; 20]) + .collect::>(); + assert_eq!(node_id_keys.len(), 1, "payout {payout:?}: {identity:?}"); + assert!( + identity.iter().all(|(id, ..)| identity + .iter() + .filter(|(other, ..)| other == id) + .count() + == 1), + "payout {payout:?}: {identity:?}" + ); + } + } + + /// Only an added key takes a new id, so an operator identity whose last key id is the + /// largest one can still have its payout key reused or its other keys disabled. + #[test] + fn should_reconcile_without_a_new_key_id_when_no_key_is_added() { + let (platform, platform_state) = platform_with_operator(Some([0x51; 20]), &[], false); + let key = |id: KeyID, data: [u8; 20], disabled: bool| -> IdentityPublicKey { + IdentityPublicKeyV0 { + id, + key_type: KeyType::ECDSA_HASH160, + purpose: Purpose::TRANSFER, + security_level: SecurityLevel::CRITICAL, + read_only: true, + data: BinaryData::new(data.to_vec()), + disabled_at: disabled.then_some(1), + contract_bounds: None, + } + .into() + }; + let identity: Identity = IdentityV0 { + id: operator_identifier(), + public_keys: BTreeMap::from([ + (0, key(0, [0x51; 20], false)), + (KeyID::MAX, key(KeyID::MAX, [0x52; 20], true)), + ]), + balance: 0, + revision: 0, + } + .into(); + platform + .drive + .add_new_identity( + identity, + true, + &BlockInfo::default(), + true, + None, + PlatformVersion::latest(), + ) + .expect("expected to add the operator identity"); + + update( + &platform, + &platform_state, + None, + Some(Some([0x52; 20])), + None, + ) + .expect("expected to reuse the payout key"); + assert_eq!( + keys(&platform), + [ + (Purpose::TRANSFER, vec![0x51; 20], true), + (Purpose::TRANSFER, vec![0x52; 20], false), + ] + ); + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/mod.rs index a32101611ee..94b69b324de 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/mod.rs @@ -1,6 +1,7 @@ mod update_state_masternode_list; mod v0; mod v1; +mod v2; use crate::error::execution::ExecutionError; use crate::error::Error; @@ -70,9 +71,18 @@ where transaction, platform_version, ), + 2 => self.update_masternode_list_v2( + platform_state, + block_platform_state, + core_block_height, + is_init_chain, + block_info, + transaction, + platform_version, + ), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "update_masternode_list".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, })), } @@ -1119,12 +1129,12 @@ mod test { /// A Core diff with no field the client decodes set: the fields a test's diff carries are /// added on top of this. - fn empty_state_diff() -> DMNStateDiff { + pub(super) fn empty_state_diff() -> DMNStateDiff { serde_json::from_str("{}").expect("an empty Core state diff") } /// A Regular masternode named by `id` with an operator payout and an operator reward. - fn masternode_with_operator_payout(id: u8) -> MasternodeListItem { + pub(super) fn masternode_with_operator_payout(id: u8) -> MasternodeListItem { MasternodeListItem { node_type: MasternodeType::Regular, pro_tx_hash: ProTxHash::from_byte_array([id; 32]), @@ -1233,7 +1243,7 @@ mod test { } /// The keys of a masternode's operator identity, with their purpose, data and disabled state. - fn operator_identity_keys( + pub(super) fn operator_identity_keys( platform: &TempPlatform, transaction: Option<&Transaction>, pro_tx_hash: &ProTxHash, @@ -1264,16 +1274,16 @@ mod test { .collect() } - /// The masternode list generations so far (v0 and v1) processed Core diffs without their - /// operator payout: operator payouts reached Platform only in full masternode states, and the - /// stored list and the operator identities' keys follow from that. A diff that sets or clears - /// an operator payout must leave both as they were, whether the operator key stays or - /// changes. + /// The masternode list generations v0 and v1 processed Core diffs without their operator + /// payout: operator payouts reached Platform only in full masternode states, and the stored + /// list and the operator identities' keys follow from that. A diff that sets or clears an + /// operator payout must leave both as they were, whether the operator key stays or changes. + /// From protocol version 15 the payout applies (`operator_payout_tests`). #[test] fn should_keep_operator_payouts_and_keys_when_a_diff_carries_an_operator_payout() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), - PlatformVersion::latest(), + PlatformVersion::get(14).expect("expected protocol version 14"), ] { let context = format!("protocol version {}", platform_version.protocol_version); let payout_set = masternode_with_operator_payout(0x61); @@ -1377,8 +1387,8 @@ mod test { } /// Core prints `PoSeRevivedHeight` only in a diff that spans the revive. The masternode list - /// generations so far (v0 and v1) always applied a diff with the revived height replaced by - /// the diff's, so a revived masternode's next diff clears it; their stored lists and + /// generations v0 and v1 always applied a diff with the revived height replaced by the + /// diff's, so a revived masternode's next diff clears it; their stored lists and /// validator sets follow from that, and they keep it whatever the Core RPC client's own diff /// application does. The payment-only diff of a revived evonode, alone in its block, is what /// decides v0's early return and reaches the evonode list as well as the full list. @@ -1386,7 +1396,7 @@ mod test { fn should_clear_a_revived_height_a_diff_does_not_print() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), - PlatformVersion::latest(), + PlatformVersion::get(14).expect("expected protocol version 14"), ] { let context = format!("protocol version {}", platform_version.protocol_version); let pose_heights = |state: &PlatformState, pro_tx_hash: &ProTxHash| { @@ -1466,4 +1476,6 @@ mod test { #[cfg(test)] mod operator_key_encoding_tests; #[cfg(test)] +mod operator_payout_tests; +#[cfg(test)] mod versioning_tests; diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_key_encoding_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_key_encoding_tests.rs index 1fd9324bd58..23225ef1e28 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_key_encoding_tests.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_key_encoding_tests.rs @@ -12,27 +12,32 @@ use crate::config::PlatformConfig; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::{PlatformState, PlatformStateV0Methods}; -use crate::rpc::core::MockCoreRPCLike; +use crate::rpc::core::{CoreRPCLike, MockCoreRPCLike, V24_COINBASE_PAYLOAD_VERSION}; use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; use dpp::block::block_info::BlockInfo; +use dpp::block::extended_block_info::v0::ExtendedBlockInfoV0; use dpp::bls::SecretKey as BlsSecretKey; use dpp::dashcore::address::NetworkUnchecked; use dpp::dashcore::hashes::Hash; use dpp::dashcore::{Address, ProTxHash}; -use dpp::dashcore_rpc::json::{DMNStateDiff, MasternodeListDiff, MasternodeListItem}; +use dpp::dashcore_rpc::json::{ + DMNState, DMNStateDiff, ExtendedQuorumListResult, MasternodeListDiff, MasternodeListItem, +}; use dpp::identifier::MasternodeIdentifiers; use dpp::identity::accessors::IdentityGettersV0; use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; use dpp::identity::{KeyType, Purpose}; use dpp::prelude::Identifier; use dpp::version::PlatformVersion; +use drive::drive::system::core_fork_activations::CoreFork; use drive::grovedb::Transaction; use rand::rngs::StdRng; use rand::SeedableRng; use serde_json::json; -use std::collections::BTreeMap; +use std::collections::{BTreeMap, HashMap}; use std::path::PathBuf; use std::str::FromStr; +use std::sync::{Arc, Mutex}; /// The first Core height of the recording where DEPLOYMENT_V24 is active. const V24_ACTIVATION_HEIGHT: u32 = 400; @@ -152,9 +157,37 @@ fn platform_with_core( 3 })) }); + // Up to protocol version 14 the list update never asks Core for a masternode's state: no + // answer is given, so a call fails the test. + if platform_version + .drive_abci + .methods + .core_based_updates + .update_masternode_list + >= 2 + { + platform + .core_rpc + .expect_get_masternode_state() + .returning(|pro_tx_hash, _| Ok(recorded_state(pro_tx_hash))); + } platform } +/// Core's state of a recorded masternode once its operator key changed: the listed state, with +/// no operator payout for MN_L3, whose owner set another operator key, which empties it. MN_L1 +/// and MN_L4 list no payout, and MN_L2's re-encoded key kept its own. +fn recorded_state(masternode: &ProTxHash) -> DMNState { + let mut state = recorded_masternodes() + .remove(masternode) + .unwrap_or_else(|| panic!("no recorded masternode {masternode}")) + .state; + if *masternode == pro_tx_hash(MN_L3) { + state.operator_payout_address = None; + } + state +} + /// The recorded diffs one Core height at a time, from the list at 399 to MN_L4's revocation. fn recorded_steps() -> BTreeMap { let mut steps = BTreeMap::from([( @@ -170,10 +203,53 @@ fn recorded_steps() -> BTreeMap { steps } +/// Records Dash Core's V24 as active at `core_height` when the coinbase there says so and it is +/// not recorded yet, as `update_core_info` 1 does before it updates the masternode list. Only for +/// the protocol versions selecting that generation, whose masternode list update reads the record +/// instead of asking Core. +fn record_v24_as_update_core_info_does( + platform: &TempPlatform, + transaction: &Transaction, + platform_version: &PlatformVersion, + core_height: u32, +) { + if platform_version + .drive_abci + .methods + .core_based_updates + .update_core_info + == 0 + || platform + .drive + .fetch_core_fork_activation_height(CoreFork::V24, Some(transaction), platform_version) + .expect("expected to read the V24 record") + .is_some() + { + return; + } + let coinbase_payload_version = platform + .core_rpc + .get_coinbase_payload_version(core_height) + .expect("expected Core's coinbase") + .expect("expected a coinbase payload"); + if coinbase_payload_version >= V24_COINBASE_PAYLOAD_VERSION { + platform + .drive + .store_core_fork_activation_height( + CoreFork::V24, + core_height, + Some(transaction), + platform_version, + ) + .expect("expected to record V24"); + } +} + /// Runs one masternode list update per Core height of `list_diffs`, in order, as Drive runs one -/// per block, the first one from scratch as at init chain. With `reload_after`, the platform state -/// is stored and fetched back after that height, as a restarted node continues. Returns the state -/// after the last update. +/// per block, the first one from scratch as at init chain, each after the Core fork record step +/// of `update_core_info` (`record_v24_as_update_core_info_does`). With `reload_after`, the +/// platform state is stored and fetched back after that height, as a restarted node continues. +/// Returns the state after the last update. fn run_core_heights( platform: &TempPlatform, transaction: &Transaction, @@ -185,6 +261,7 @@ fn run_core_heights( let mut block_state = platform.state.load().as_ref().clone(); for (index, core_height) in core_heights.into_iter().enumerate() { let block_height = index as u64 + 1; + record_v24_as_update_core_info_does(platform, transaction, platform_version, core_height); platform .update_masternode_list( committed.as_ref(), @@ -275,6 +352,7 @@ fn voting_key_hash(address: &str) -> [u8; 20] { fn should_keep_the_operator_identity_of_a_legacy_key_core_re_encodes() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), + PlatformVersion::get(14).expect("expected protocol version 14"), PlatformVersion::latest(), ] { let version = platform_version.protocol_version; @@ -412,6 +490,7 @@ fn should_keep_the_operator_identity_of_a_legacy_key_core_re_encodes() { fn should_switch_the_operator_identity_on_a_re_encoded_key_while_v24_is_not_active() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), + PlatformVersion::get(14).expect("expected protocol version 14"), PlatformVersion::latest(), ] { let version = platform_version.protocol_version; @@ -489,6 +568,7 @@ fn should_reach_the_kept_operator_identity_after_a_re_encode_and_a_restart() { }; for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), + PlatformVersion::get(14).expect("expected protocol version 14"), PlatformVersion::latest(), ] { let version = platform_version.protocol_version; @@ -566,7 +646,8 @@ fn should_reach_the_kept_operator_identity_after_a_re_encode_and_a_restart() { /// recorded ProUpRegTx step, here with an operator payout added. Platform drops both from the /// entry. MN_L2 keeps its legacy key bytes, the operator payout of its full state, and its /// operator identity with every key, the payout key included; and the blocks end exactly as they -/// do without the diff's operator payout. +/// do without the diff's operator payout. Protocol version 15 applies the payout instead +/// (`should_apply_the_operator_payout_of_a_diff_that_re_encodes_the_operator_key`). #[test] fn should_ignore_the_operator_payout_of_a_diff_that_re_encodes_the_operator_key() { let mn_l2 = pro_tx_hash(MN_L2); @@ -575,7 +656,7 @@ fn should_ignore_the_operator_payout_of_a_diff_that_re_encodes_the_operator_key( assert!(listed_payout.is_some_and(|payout| payout != diff_payout)); for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), - PlatformVersion::latest(), + PlatformVersion::get(14).expect("expected protocol version 14"), ] { let version = platform_version.protocol_version; let mut outcomes = vec![]; @@ -657,6 +738,78 @@ fn should_ignore_the_operator_payout_of_a_diff_that_re_encodes_the_operator_key( } } +/// MN_L2's recorded ProUpRegTx step with an operator payout added: Platform drops the re-encoded +/// key, so MN_L2 keeps its legacy key bytes and its operator identity, and applies the payout to +/// that identity. The listed payout's key is disabled and the diff's added; the operator key stays +/// enabled, and no identity is created for the basic bytes. +#[test] +fn should_apply_the_operator_payout_of_a_diff_that_re_encodes_the_operator_key() { + let platform_version = PlatformVersion::latest(); + let mn_l2 = pro_tx_hash(MN_L2); + let diff_payout = [0x81; 20]; + let listed_payout = recorded_masternodes()[&mn_l2] + .state + .operator_payout_address + .expect("expected MN_L2 to list an operator payout"); + let mut list_diffs = recorded_steps(); + let (_, entry) = list_diffs + .get_mut(&(V24_ACTIVATION_HEIGHT + 2)) + .and_then(|diff| { + diff.updated_mns + .iter_mut() + .find(|(pro_tx_hash, _)| *pro_tx_hash == mn_l2) + }) + .expect("expected MN_L2's recorded re-encode"); + assert_eq!(entry.pub_key_operator, Some(bytes(MN_L2_BASIC_KEY))); + entry.operator_payout_address = Some(Some(diff_payout)); + let core_heights: Vec = list_diffs.keys().copied().collect(); + let platform = platform_with_core(platform_version, list_diffs, V24_ACTIVATION_HEIGHT); + let transaction = platform.drive.grove.start_transaction(); + let state = run_core_heights( + &platform, + &transaction, + platform_version, + core_heights, + None, + ); + + let stored = &state.full_masternode_list()[&mn_l2].state; + assert_eq!(stored.pub_key_operator, bytes(MN_L2_LEGACY_KEY)); + assert_eq!(stored.operator_payout_address, Some(diff_payout)); + let operator_keys = identity_keys( + &platform, + &transaction, + platform_version, + operator_identifier(&mn_l2, &bytes(MN_L2_LEGACY_KEY)), + ) + .expect("expected MN_L2's operator identity"); + assert_eq!( + bls_key_disabled(&operator_keys, &bytes(MN_L2_LEGACY_KEY)), + Some(false) + ); + let transfer_keys = operator_keys + .iter() + .filter(|(_, purpose, ..)| *purpose == Purpose::TRANSFER) + .map(|(_, _, data, disabled)| (data.clone(), *disabled)) + .collect::>(); + assert_eq!( + transfer_keys, + [ + (listed_payout.to_vec(), true), + (diff_payout.to_vec(), false) + ] + ); + assert_eq!( + identity_keys( + &platform, + &transaction, + platform_version, + operator_identifier(&mn_l2, &bytes(MN_L2_BASIC_KEY)) + ), + None + ); +} + /// The masternode for the synthetic steps: MN_L1 as recorded, with operator key `key`. fn masternode_with_key(key: Vec) -> MasternodeListItem { let mut masternode = recorded_masternodes() @@ -728,6 +881,7 @@ fn run_operator_key_steps( fn should_apply_every_reported_operator_key_before_v24() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), + PlatformVersion::get(14).expect("expected protocol version 14"), PlatformVersion::latest(), ] { let version = platform_version.protocol_version; @@ -783,6 +937,7 @@ fn should_apply_every_reported_operator_key_before_v24() { fn should_keep_the_operator_identity_for_a_negated_key_once_v24_is_active() { for platform_version in [ PlatformVersion::get(13).expect("expected protocol version 13"), + PlatformVersion::get(14).expect("expected protocol version 14"), PlatformVersion::latest(), ] { let version = platform_version.protocol_version; @@ -823,3 +978,168 @@ fn should_keep_the_operator_identity_for_a_negated_key_once_v24_is_active() { ); } } + +/// From protocol version 15 the masternode list update reads whether V24 is active from the Core +/// fork record that `update_core_info` writes before it, at the same chain locked height, and +/// never asks Core about it. MN_L2's recorded re-encode (402) keeps its operator identity and its +/// payout when V24 was recorded in an earlier block or in this very block, and is a key change +/// when Core reports V24 inactive, which takes the payout Core reports for MN_L2. Core is asked +/// about the coinbase only by the record step, once per block that moves the chain locked height +/// until V24 is recorded. +#[test] +fn should_take_v24_from_the_core_fork_record() { + let platform_version = PlatformVersion::latest(); + let mn_l2 = pro_tx_hash(MN_L2); + let listed_payout = recorded_masternodes()[&mn_l2].state.operator_payout_address; + assert!(listed_payout.is_some()); + for (case, v24_from, kept, expected_reads) in [ + ("recorded earlier", 400, true, vec![399, 400]), + ( + "recorded in this block", + 402, + true, + vec![399, 400, 401, 402], + ), + ("not active", u32::MAX, false, vec![399, 400, 401, 402]), + ] { + let list_diffs = BTreeMap::from( + [(1, 399), (399, 400), (400, 401), (401, 402)] + .map(|(base, core_height)| (core_height, recorded_list_diff(base, core_height))), + ); + let mut platform = TestPlatformBuilder::new() + .with_config(PlatformConfig::default()) + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_genesis_state(); + // Without the default answers, which would shadow the ones set here. + platform.core_rpc.checkpoint(); + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |_, core_height| Ok(list_diffs[&core_height].clone())); + let quorum_types = [ + platform.config.validator_set.quorum_type, + platform.config.chain_lock.quorum_type, + platform.config.instant_lock.quorum_type, + ]; + platform + .core_rpc + .expect_get_quorum_listextended() + .returning(move |_| { + Ok(ExtendedQuorumListResult { + quorums_by_type: quorum_types + .iter() + .map(|quorum_type| (*quorum_type, Default::default())) + .collect::>(), + }) + }); + let reads = Arc::new(Mutex::new(vec![])); + let recorded_reads = Arc::clone(&reads); + platform + .core_rpc + .expect_get_coinbase_payload_version() + .returning(move |core_height| { + recorded_reads + .lock() + .expect("expected the coinbase reads") + .push(core_height); + Ok(Some(if core_height >= v24_from { 4 } else { 3 })) + }); + + platform + .core_rpc + .expect_get_masternode_state() + .returning(|pro_tx_hash, _| Ok(recorded_state(pro_tx_hash))); + + let transaction = platform.drive.grove.start_transaction(); + let mut committed: Option = None; + let mut block_state = platform.state.load().as_ref().clone(); + let mut previous_core_height = None; + for (height, core_height) in (1..).zip([399, 400, 401, 402]) { + if let Some(previous_core_height) = previous_core_height { + block_state.set_last_committed_block_info(Some( + ExtendedBlockInfoV0 { + basic_info: BlockInfo { + height: height - 1, + core_height: previous_core_height, + ..Default::default() + }, + app_hash: [0; 32], + quorum_hash: [0; 32], + block_id_hash: [0; 32], + proposer_pro_tx_hash: [0; 32], + signature: [0; 96], + round: 0, + } + .into(), + )); + } + platform + .update_core_info( + committed.as_ref(), + &mut block_state, + core_height, + committed.is_none(), + &BlockInfo { + height, + core_height, + time_ms: height, + ..Default::default() + }, + &transaction, + platform_version, + ) + .expect("expected to update the core info"); + committed = Some(block_state.clone()); + previous_core_height = Some(core_height); + } + + let stored = &block_state.full_masternode_list()[&mn_l2].state; + let legacy_identity = identity_keys( + &platform, + &transaction, + platform_version, + operator_identifier(&mn_l2, &bytes(MN_L2_LEGACY_KEY)), + ) + .expect("expected MN_L2's operator identity"); + let basic_identity = identity_keys( + &platform, + &transaction, + platform_version, + operator_identifier(&mn_l2, &bytes(MN_L2_BASIC_KEY)), + ); + if kept { + assert_eq!(stored.pub_key_operator, bytes(MN_L2_LEGACY_KEY), "{case}"); + assert_eq!(stored.operator_payout_address, listed_payout, "{case}"); + assert!( + legacy_identity.iter().all(|(.., disabled)| !disabled), + "{case}: {legacy_identity:?}" + ); + assert_eq!(basic_identity, None, "{case}"); + } else { + // Read as a key change, which Core prints without a payout: the payout is Core's, + // which a re-encode keeps. + assert_eq!(stored.pub_key_operator, bytes(MN_L2_BASIC_KEY), "{case}"); + assert_eq!(stored.operator_payout_address, listed_payout, "{case}"); + assert_eq!( + bls_key_disabled(&legacy_identity, &bytes(MN_L2_LEGACY_KEY)), + Some(true), + "{case}" + ); + let basic_identity = basic_identity.expect("expected the basic bytes' identity"); + assert_eq!( + basic_identity + .iter() + .filter(|(_, purpose, ..)| *purpose == Purpose::TRANSFER) + .map(|(_, _, data, disabled)| (data.clone(), *disabled)) + .collect::>(), + [( + listed_payout.expect("expected MN_L2's payout").to_vec(), + false + )], + "{case}" + ); + } + assert_eq!(*reads.lock().unwrap(), expected_reads, "{case}"); + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_payout_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_payout_tests.rs new file mode 100644 index 00000000000..47436f743bf --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/operator_payout_tests.rs @@ -0,0 +1,1122 @@ +//! Operator payouts from Core's diffs, as `update_masternode_list` 2 and +//! `update_operator_identity` 1 apply them from protocol version 15. + +use super::test::{empty_state_diff, masternode_with_operator_payout, operator_identity_keys}; +use crate::config::PlatformConfig; +use crate::error::Error; +use crate::platform_types::platform_state::{PlatformState, PlatformStateV0Methods}; +use crate::platform_types::state_transitions_processing_result::StateTransitionExecutionResult; +use crate::rpc::core::MockCoreRPCLike; +use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; +use assert_matches::assert_matches; +use dpp::block::block_info::BlockInfo; +use dpp::dash_to_credits; +use dpp::dashcore::hashes::Hash; +use dpp::dashcore::ProTxHash; +use dpp::dashcore_rpc::json::{ + DMNStateDiff, MasternodeListDiff, MasternodeListItem, MasternodeType, +}; +use dpp::dashcore_rpc::Error as CoreRpcError; +use dpp::fee::Credits; +use dpp::identifier::{Identifier, MasternodeIdentifiers}; +use dpp::identity::accessors::IdentityGettersV0; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::{IdentityPublicKey, KeyType, Purpose, SecurityLevel}; +use dpp::serialization::PlatformSerializable; +use dpp::state_transition::identity_credit_withdrawal_transition::methods::{ + IdentityCreditWithdrawalTransitionMethodsV0, PreferredKeyPurposeForSigningWithdrawal, +}; +use dpp::state_transition::identity_credit_withdrawal_transition::IdentityCreditWithdrawalTransition; +use dpp::util::hash::ripemd160_sha256; +use dpp::version::PlatformVersion; +use dpp::withdrawal::Pooling; +use drive::grovedb::Transaction; +use rand::rngs::StdRng; +use rand::SeedableRng; +use simple_signer::signer::SimpleSigner; +use std::collections::BTreeSet; +use std::sync::{Arc, Mutex}; + +/// Core height of the full list the masternodes start from; diff `i` ends at the next heights. +const LIST_HEIGHT: u32 = 1000; + +/// Processes a mock Core's full list of `masternodes` at Core height 1000 and then each of +/// `diffs`, one Core height and one block each, on a platform at `platform_version`. Returns the +/// platform, with the updates committed, and the platform state after the last diff, whose +/// change trackers hold only what that diff changed. Asked for a masternode's state, Core +/// reports no operator payout, as after an operator key change that set none. +fn masternode_list_after_diffs( + platform_version: &PlatformVersion, + masternodes: Vec, + diffs: Vec, +) -> (TempPlatform, PlatformState) { + let (platform, state, _) = masternode_list_after_diffs_with_core_payouts( + platform_version, + masternodes, + diffs, + &[], + |_, _| {}, + ); + (platform, state) +} + +/// The masternode states Platform asked Core for, as (masternode, Core height). +type StateReads = Arc>>; + +/// `masternode_list_after_diffs`, with Core reporting `core_payouts` as the operator payout of +/// a masternode at a Core height, and no payout for any other, and calling `before_diffs` once +/// the full list is processed. Also returns the masternode states Platform asked Core for. +fn masternode_list_after_diffs_with_core_payouts( + platform_version: &PlatformVersion, + masternodes: Vec, + diffs: Vec, + core_payouts: &[(ProTxHash, u32, Option<[u8; 20]>)], + before_diffs: impl FnOnce(&TempPlatform, &Transaction), +) -> (TempPlatform, PlatformState, StateReads) { + let mut platform = TestPlatformBuilder::new() + .with_config(PlatformConfig::default()) + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_genesis_state(); + let full_list = MasternodeListDiff { + base_height: 0, + block_height: LIST_HEIGHT, + added_mns: masternodes, + removed_mns: vec![], + updated_mns: vec![], + }; + let diffs = diffs + .into_iter() + .zip(LIST_HEIGHT..) + .map(|(updated_mns, base_height)| MasternodeListDiff { + base_height, + block_height: base_height + 1, + added_mns: vec![], + removed_mns: vec![], + updated_mns, + }) + .collect::>(); + let core_heights = diffs + .iter() + .map(|diff| diff.block_height) + .collect::>(); + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |_base_block, block| { + Ok(if block == LIST_HEIGHT { + full_list.clone() + } else { + diffs[(block - LIST_HEIGHT - 1) as usize].clone() + }) + }); + + let core_payouts = core_payouts.to_vec(); + let reads = StateReads::default(); + let recorded_reads = Arc::clone(&reads); + platform + .core_rpc + .expect_get_masternode_state() + .returning(move |pro_tx_hash, core_height| { + recorded_reads + .lock() + .expect("expected the state reads") + .push((*pro_tx_hash, core_height)); + let mut state = masternode_with_operator_payout(0).state; + state.operator_payout_address = core_payouts + .iter() + .find(|(masternode, height, _)| masternode == pro_tx_hash && *height == core_height) + .and_then(|(_, _, payout)| *payout); + Ok(state) + }); + + let transaction = platform.drive.grove.start_transaction(); + let mut committed: Option = None; + let mut block_state = platform.state.load().as_ref().clone(); + let mut before_diffs = Some(before_diffs); + for (height, core_height) in (1..).zip([LIST_HEIGHT].into_iter().chain(core_heights)) { + block_state.mark_saved(); + platform + .update_masternode_list( + committed.as_ref(), + &mut block_state, + core_height, + committed.is_none(), + &BlockInfo { + height, + core_height, + time_ms: height, + ..Default::default() + }, + &transaction, + platform_version, + ) + .expect("expected to process Core's masternode list"); + committed = Some(block_state.clone()); + if let Some(before_diffs) = before_diffs.take() { + before_diffs(&platform, &transaction); + } + } + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected to commit"); + (platform, block_state, reads) +} + +/// The updated masternodes of one Core diff. +type DMNStateDiffs = Vec<(ProTxHash, DMNStateDiff)>; + +/// The (data, disabled) of every TRANSFER key of the operator identity of `masternode` with +/// operator key `pub_key_operator`, by key id. +fn transfer_keys( + platform: &TempPlatform, + masternode: &MasternodeListItem, + pub_key_operator: &[u8], +) -> Vec<(Vec, bool)> { + operator_identity_keys( + platform, + None, + &masternode.pro_tx_hash, + pub_key_operator, + PlatformVersion::latest(), + ) + .into_values() + .filter(|(purpose, _, _)| *purpose == Purpose::TRANSFER) + .map(|(_, data, disabled)| (data, disabled)) + .collect() +} + +fn stored_operator_payout( + state: &PlatformState, + masternode: &MasternodeListItem, +) -> Option<[u8; 20]> { + state.full_masternode_list()[&masternode.pro_tx_hash] + .state + .operator_payout_address +} + +/// An operator payout Core reports in a diff reaches the stored list and the operator identity: +/// the previous payout's key is disabled and the new payout's key added, also when the identity +/// had no payout key, as every masternode registered with an operator reward starts (a ProRegTx +/// carries no operator payout). The block upserts exactly those masternodes. +#[test] +fn should_add_the_new_operator_payout_key_to_the_operator_identity() { + let changed = masternode_with_operator_payout(0x61); + let mut first_set = masternode_with_operator_payout(0x62); + first_set.state.operator_payout_address = None; + let (platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![changed.clone(), first_set.clone()], + vec![vec![ + ( + changed.pro_tx_hash, + DMNStateDiff { + operator_payout_address: Some(Some([0x81; 20])), + ..empty_state_diff() + }, + ), + ( + first_set.pro_tx_hash, + DMNStateDiff { + operator_payout_address: Some(Some([0x82; 20])), + ..empty_state_diff() + }, + ), + ]], + ); + + assert_eq!( + transfer_keys(&platform, &changed, &changed.state.pub_key_operator), + [(vec![0x61; 20], true), (vec![0x81; 20], false)] + ); + assert_eq!( + transfer_keys(&platform, &first_set, &first_set.state.pub_key_operator), + [(vec![0x82; 20], false)] + ); + assert_eq!(stored_operator_payout(&state, &changed), Some([0x81; 20])); + assert_eq!(stored_operator_payout(&state, &first_set), Some([0x82; 20])); + assert_eq!( + state.masternode_changes.upserted, + BTreeSet::from([changed.pro_tx_hash, first_set.pro_tx_hash]) + ); +} + +/// Core clears the operator payout whenever the operator key changes and prints no field for the +/// clear, so a new operator identity gets only the payout Core reports with the new key, never +/// the stored one, which is the previous operator's. The previous operator's identity keeps its +/// payout key, so its operator can still withdraw what was sent to it. +#[test] +fn should_not_give_a_new_operator_identity_the_previous_operators_payout_key() { + let reset = masternode_with_operator_payout(0x63); + let reset_and_set = masternode_with_operator_payout(0x64); + let (platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![reset.clone(), reset_and_set.clone()], + vec![vec![ + ( + reset.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![0x73; 48]), + ..empty_state_diff() + }, + ), + ( + reset_and_set.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![0x74; 48]), + operator_payout_address: Some(Some([0x84; 20])), + ..empty_state_diff() + }, + ), + ]], + ); + + assert_eq!(transfer_keys(&platform, &reset, &[0x73; 48]), []); + assert_eq!(stored_operator_payout(&state, &reset), None); + assert_eq!( + transfer_keys(&platform, &reset_and_set, &[0x74; 48]), + [(vec![0x84; 20], false)] + ); + assert_eq!( + stored_operator_payout(&state, &reset_and_set), + Some([0x84; 20]) + ); + for masternode in [&reset, &reset_and_set] { + assert_eq!( + transfer_keys(&platform, masternode, &masternode.state.pub_key_operator), + [( + masternode.state.operator_payout_address.unwrap().to_vec(), + false + )], + "the previous operator's identity of {}", + masternode.pro_tx_hash + ); + } +} + +/// An operator key change of `masternode` to `pub_key_operator`, with `payout` printed if any. +fn operator_key_change( + masternode: &MasternodeListItem, + pub_key_operator: &[u8], + payout: Option<[u8; 20]>, +) -> (ProTxHash, DMNStateDiff) { + ( + masternode.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(pub_key_operator.to_vec()), + operator_payout_address: payout.map(Some), + ..empty_state_diff() + }, + ) +} + +/// An operator key that comes back lands on the identity its earlier operator left, which still +/// holds that operator's payout key. The key change leaves its TRANSFER keys as they are, with or +/// without a payout Core reports with it: the owner alone authorizes a key change, so that payout +/// need not be the returning operator's own choice, and the earlier operator must still be able +/// to withdraw what was sent to it. The stored payout follows Core. The next payout change Core +/// reports while the masternode stays on that identity reconciles its keys as usual. +#[test] +fn should_leave_the_payout_keys_of_an_identity_a_masternode_switches_back_to_until_its_next_payout_change( +) { + let without_payout = masternode_with_operator_payout(0x65); + let with_payout = masternode_with_operator_payout(0x66); + let (platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![without_payout.clone(), with_payout.clone()], + vec![ + vec![ + operator_key_change(&without_payout, &[0x75; 48], Some([0x85; 20])), + operator_key_change(&with_payout, &[0x76; 48], Some([0x86; 20])), + ], + vec![ + operator_key_change(&without_payout, &[0x65; 48], None), + operator_key_change(&with_payout, &[0x66; 48], Some([0x96; 20])), + ], + vec![( + with_payout.pro_tx_hash, + DMNStateDiff { + operator_payout_address: Some(Some([0xa6; 20])), + ..empty_state_diff() + }, + )], + ], + ); + + assert_eq!( + transfer_keys( + &platform, + &without_payout, + &without_payout.state.pub_key_operator + ), + [(vec![0x65; 20], false)] + ); + assert_eq!(stored_operator_payout(&state, &without_payout), None); + assert_eq!( + transfer_keys(&platform, &with_payout, &with_payout.state.pub_key_operator), + [(vec![0x66; 20], true), (vec![0xa6; 20], false)] + ); + assert_eq!( + stored_operator_payout(&state, &with_payout), + Some([0xa6; 20]) + ); + // The intermediate operators keep their payout keys. + assert_eq!( + transfer_keys(&platform, &without_payout, &[0x75; 48]), + [(vec![0x85; 20], false)] + ); + assert_eq!( + transfer_keys(&platform, &with_payout, &[0x76; 48]), + [(vec![0x86; 20], false)] + ); +} + +/// The credits, 0.5 Dash, an operator identity holds in the funded scenarios. +const ACCRUED: Credits = 50_000_000_000; + +/// Masternode `id`, whose operator payout is the hash of a key the test can sign with, and that +/// key's private key. +fn masternode_with_signable_operator_payout(id: u8) -> (MasternodeListItem, [u8; 32]) { + let mut rng = StdRng::seed_from_u64(id as u64); + let (payout_key, private_key) = IdentityPublicKey::random_key_with_known_attributes( + 0, + &mut rng, + Purpose::TRANSFER, + SecurityLevel::CRITICAL, + KeyType::ECDSA_HASH160, + None, + PlatformVersion::latest(), + ) + .expect("expected a payout key"); + let mut masternode = masternode_with_operator_payout(id); + masternode.state.operator_payout_address = Some( + payout_key + .data() + .as_slice() + .try_into() + .expect("expected a 20-byte payout"), + ); + (masternode, private_key) +} + +/// `masternode_list_after_diffs` at the latest protocol version for `masternode` alone, with +/// `ACCRUED` credited to its first operator identity once the full list is processed. +fn funded_operator_after_diffs( + masternode: &MasternodeListItem, + diffs: Vec, +) -> (TempPlatform, PlatformState) { + let platform_version = PlatformVersion::latest(); + let identity_id = Identifier::create_operator_identifier( + masternode.pro_tx_hash.as_byte_array(), + &masternode.state.pub_key_operator, + ); + let (platform, state, _) = masternode_list_after_diffs_with_core_payouts( + platform_version, + vec![masternode.clone()], + diffs, + &[], + |platform, transaction| { + platform + .drive + .add_to_system_credits(ACCRUED, Some(transaction), platform_version) + .expect("expected to add to system credits"); + platform + .drive + .add_to_identity_balance( + identity_id.to_buffer(), + ACCRUED, + &BlockInfo::default(), + true, + Some(transaction), + platform_version, + ) + .expect("expected to credit the operator identity"); + }, + ); + (platform, state) +} + +/// Asserts that the first operator identity of `masternode` still holds `ACCRUED`, and that a +/// withdrawal from it signed with its payout key, whose private key is `payout_private_key`, +/// executes. +async fn assert_first_operator_withdraws_with_its_payout_key( + platform: &TempPlatform, + masternode: &MasternodeListItem, + payout_private_key: [u8; 32], +) { + let platform_version = PlatformVersion::latest(); + let payout = masternode + .state + .operator_payout_address + .expect("expected the first operator's payout"); + let identity = platform + .drive + .fetch_full_identity( + Identifier::create_operator_identifier( + masternode.pro_tx_hash.as_byte_array(), + &masternode.state.pub_key_operator, + ) + .to_buffer(), + None, + platform_version, + ) + .expect("expected to fetch the operator identity") + .expect("expected the operator identity"); + assert_eq!(identity.balance(), ACCRUED); + let payout_key = identity + .public_keys() + .values() + .find(|key| key.purpose() == Purpose::TRANSFER && key.data().as_slice() == payout) + .cloned() + .expect("expected the payout key"); + let mut signer = SimpleSigner::default(); + signer.add_identity_public_key(payout_key.clone(), payout_private_key); + let withdrawal = IdentityCreditWithdrawalTransition::try_from_identity( + &identity, + None, + dash_to_credits!(0.1), + Pooling::Never, + 1, + 0, + signer, + Some(&payout_key), + PreferredKeyPurposeForSigningWithdrawal::TransferOnly, + 1, + platform_version, + None, + ) + .await + .expect("expected a withdrawal signed with the payout key"); + + let transaction = platform.drive.grove.start_transaction(); + let processing_result = platform + .platform + .process_raw_state_transitions( + &[withdrawal + .serialize_to_bytes() + .expect("expected to serialize the withdrawal")], + &platform.state.load(), + &BlockInfo::default_with_time(1_200_001_000), + &transaction, + platform_version, + false, + None, + ) + .expect("expected to process the withdrawal"); + assert_matches!( + processing_result.execution_results().as_slice(), + [StateTransitionExecutionResult::SuccessfulExecution { .. }] + ); +} + +/// Core authenticates an operator key change with the owner's key alone, so an owner can rotate +/// the operator key away from operator A, back to A without a payout, and away again, all without +/// A. A must still be able to withdraw the credits it accrued: the return without a payout leaves +/// the TRANSFER keys of A's identity as they are, so its payout key stays enabled after the owner +/// moves on, and a withdrawal signed with it goes through. +#[tokio::test] +async fn should_keep_a_returning_operators_payout_key_when_the_owner_switches_back_without_a_payout( +) { + let (masternode, payout_private_key) = masternode_with_signable_operator_payout(0x78); + let operator_a = masternode.state.pub_key_operator.clone(); + let operator_b = [0x88; 48]; + + let (platform, state) = funded_operator_after_diffs( + &masternode, + vec![ + vec![operator_key_change(&masternode, &operator_b, None)], + vec![operator_key_change(&masternode, &operator_a, None)], + vec![operator_key_change(&masternode, &operator_b, None)], + ], + ); + + assert_eq!( + state.full_masternode_list()[&masternode.pro_tx_hash] + .state + .pub_key_operator, + operator_b + ); + assert_eq!( + transfer_keys(&platform, &masternode, &operator_a), + [( + masternode.state.operator_payout_address.unwrap().to_vec(), + false + )] + ); + assert_first_operator_withdraws_with_its_payout_key(&platform, &masternode, payout_private_key) + .await; +} + +/// The owner alone authorizes an operator key change, so the payout Core reports with a switch +/// back to operator A need not be A's own choice. The switch back leaves the TRANSFER keys of +/// A's identity as they are: no key for that payout is added, A's payout key stays enabled, and a +/// withdrawal signed with it goes through. The stored payout follows Core. +#[tokio::test] +async fn should_keep_a_returning_operators_payout_key_when_the_owner_switches_back_with_a_payout() { + let (masternode, payout_private_key) = masternode_with_signable_operator_payout(0x79); + let operator_a = masternode.state.pub_key_operator.clone(); + + let (platform, state) = funded_operator_after_diffs( + &masternode, + vec![ + vec![operator_key_change(&masternode, &[0x89; 48], None)], + vec![operator_key_change( + &masternode, + &operator_a, + Some([0x99; 20]), + )], + ], + ); + + assert_eq!( + state.full_masternode_list()[&masternode.pro_tx_hash] + .state + .pub_key_operator, + operator_a + ); + assert_eq!( + stored_operator_payout(&state, &masternode), + Some([0x99; 20]) + ); + assert_eq!( + transfer_keys(&platform, &masternode, &operator_a), + [( + masternode.state.operator_payout_address.unwrap().to_vec(), + false + )] + ); + assert_first_operator_withdraws_with_its_payout_key(&platform, &masternode, payout_private_key) + .await; +} + +/// Core prints the stored payout again when the stored one went stale through a clear or a reset +/// Core does not print. The identity already holds that payout's key, so the update leaves it as +/// it is: adding a second key with the same data would fail on every node and halt the chain. +/// A payout that comes back after another one re-enables its key. +#[test] +fn should_not_halt_when_a_diff_re_sends_the_stored_operator_payout() { + let re_sent = masternode_with_operator_payout(0x67); + let back = masternode_with_operator_payout(0x68); + let payout = |masternode: &MasternodeListItem, payout: u8| { + ( + masternode.pro_tx_hash, + DMNStateDiff { + operator_payout_address: Some(Some([payout; 20])), + ..empty_state_diff() + }, + ) + }; + let (platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![re_sent.clone(), back.clone()], + vec![ + vec![payout(&re_sent, 0x67), payout(&back, 0x88)], + vec![payout(&re_sent, 0x67), payout(&back, 0x68)], + ], + ); + + assert_eq!( + transfer_keys(&platform, &re_sent, &re_sent.state.pub_key_operator), + [(vec![0x67; 20], false)] + ); + assert_eq!(stored_operator_payout(&state, &re_sent), Some([0x67; 20])); + assert_eq!( + transfer_keys(&platform, &back, &back.state.pub_key_operator), + [(vec![0x68; 20], false), (vec![0x88; 20], true)] + ); + assert_eq!(stored_operator_payout(&state, &back), Some([0x68; 20])); +} + +/// Core prints `PoSeRevivedHeight` only in a diff that spans the revive. The stored list keeps +/// the revived height through later diffs that do not print it, in the full list and in the +/// evonode list. +#[test] +fn should_keep_the_revived_height_a_diff_does_not_print() { + let mut revived = masternode_with_operator_payout(0x69); + revived.node_type = MasternodeType::Evo; + revived.state.platform_node_id = Some([0x69; 20]); + #[allow(deprecated)] + { + revived.state.legacy_platform_p2p_port = Some(26656); + revived.state.legacy_platform_http_port = Some(443); + } + revived.state.pose_revived_height = Some(900); + let (_platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![revived.clone()], + vec![vec![( + revived.pro_tx_hash, + DMNStateDiff { + last_paid_height: Some(1001), + ..empty_state_diff() + }, + )]], + ); + + for list in [state.full_masternode_list(), state.hpmn_masternode_list()] { + assert_eq!( + list[&revived.pro_tx_hash].state.pose_revived_height, + Some(900) + ); + } +} + +/// An evonode `id` with platform node id `[id; 20]`. +fn evonode(id: u8) -> MasternodeListItem { + let mut evonode = masternode_with_operator_payout(id); + evonode.node_type = MasternodeType::Evo; + evonode.state.platform_node_id = Some([id ^ 0x0f; 20]); + #[allow(deprecated)] + { + evonode.state.legacy_platform_p2p_port = Some(26656); + evonode.state.legacy_platform_http_port = Some(443); + } + evonode +} + +/// The (data, disabled) of every key of the operator identity of `masternode` with operator key +/// `pub_key_operator` whose purpose is `purpose`, by key id. +fn keys_with_purpose( + platform: &TempPlatform, + masternode: &MasternodeListItem, + pub_key_operator: &[u8], + purpose: Purpose, +) -> Vec<(Vec, bool)> { + operator_identity_keys( + platform, + None, + &masternode.pro_tx_hash, + pub_key_operator, + PlatformVersion::latest(), + ) + .into_values() + .filter(|(key_purpose, _, _)| *key_purpose == purpose) + .map(|(_, data, disabled)| (data, disabled)) + .collect() +} + +/// An operator payout and a platform node id are any 20 bytes Core accepts, so an operator can +/// make the payout equal the platform node id or the hash of the operator key, make the node id +/// equal the payout, or set as payout a hash that was the node id before. Each such key is added +/// and holds the authority of its kind: the payout's TRANSFER key is enabled, so the operator can +/// still withdraw, and the block does not fail on the identity already holding a key of that +/// hash. A later change of both the node id and the payout goes through. +#[test] +fn should_add_an_operator_payout_or_node_id_key_that_repeats_another_keys_hash() { + let payout_is_node_id = evonode(0x6a); + let payout_is_operator_key_hash = evonode(0x6b); + let node_id_is_payout = evonode(0x6c); + let payout_was_node_id = evonode(0x6f); + let node_id = |masternode: &MasternodeListItem| { + masternode + .state + .platform_node_id + .expect("expected an evonode") + }; + let payout = |masternode: &MasternodeListItem| { + masternode + .state + .operator_payout_address + .expect("expected an operator payout") + }; + let operator_key_hash = ripemd160_sha256(&payout_is_operator_key_hash.state.pub_key_operator); + let earlier_node_id = [0x9f; 20]; + let entry = + |masternode: &MasternodeListItem, diff: DMNStateDiff| (masternode.pro_tx_hash, diff); + let (platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![ + payout_is_node_id.clone(), + payout_is_operator_key_hash.clone(), + node_id_is_payout.clone(), + payout_was_node_id.clone(), + ], + vec![ + vec![ + entry( + &payout_is_node_id, + DMNStateDiff { + operator_payout_address: Some(Some(node_id(&payout_is_node_id))), + ..empty_state_diff() + }, + ), + entry( + &payout_is_operator_key_hash, + DMNStateDiff { + operator_payout_address: Some(Some(operator_key_hash)), + ..empty_state_diff() + }, + ), + entry( + &node_id_is_payout, + DMNStateDiff { + platform_node_id: Some(payout(&node_id_is_payout)), + ..empty_state_diff() + }, + ), + entry( + &payout_was_node_id, + DMNStateDiff { + platform_node_id: Some(earlier_node_id), + ..empty_state_diff() + }, + ), + ], + vec![ + entry( + &payout_is_node_id, + DMNStateDiff { + platform_node_id: Some([0x7a; 20]), + operator_payout_address: Some(Some([0x8a; 20])), + ..empty_state_diff() + }, + ), + entry( + &payout_was_node_id, + DMNStateDiff { + platform_node_id: Some([0x7f; 20]), + ..empty_state_diff() + }, + ), + ], + vec![entry( + &payout_was_node_id, + DMNStateDiff { + operator_payout_address: Some(Some(earlier_node_id)), + ..empty_state_diff() + }, + )], + ], + ); + + let key = &payout_is_node_id.state.pub_key_operator; + assert_eq!( + keys_with_purpose(&platform, &payout_is_node_id, key, Purpose::TRANSFER), + [ + (payout(&payout_is_node_id).to_vec(), true), + (node_id(&payout_is_node_id).to_vec(), true), + (vec![0x8a; 20], false) + ] + ); + assert_eq!( + keys_with_purpose(&platform, &payout_is_node_id, key, Purpose::SYSTEM), + [ + (key.clone(), false), + (node_id(&payout_is_node_id).to_vec(), true), + (vec![0x7a; 20], false) + ] + ); + assert_eq!( + stored_operator_payout(&state, &payout_is_node_id), + Some([0x8a; 20]) + ); + + let key = &payout_is_operator_key_hash.state.pub_key_operator; + assert_eq!( + keys_with_purpose( + &platform, + &payout_is_operator_key_hash, + key, + Purpose::TRANSFER + ), + [ + (payout(&payout_is_operator_key_hash).to_vec(), true), + (operator_key_hash.to_vec(), false) + ] + ); + + let key = &node_id_is_payout.state.pub_key_operator; + assert_eq!( + keys_with_purpose(&platform, &node_id_is_payout, key, Purpose::TRANSFER), + [(payout(&node_id_is_payout).to_vec(), false)] + ); + assert_eq!( + keys_with_purpose(&platform, &node_id_is_payout, key, Purpose::SYSTEM), + [ + (key.clone(), false), + (node_id(&node_id_is_payout).to_vec(), true), + (payout(&node_id_is_payout).to_vec(), false) + ] + ); + + let key = &payout_was_node_id.state.pub_key_operator; + assert_eq!( + keys_with_purpose(&platform, &payout_was_node_id, key, Purpose::TRANSFER), + [ + (payout(&payout_was_node_id).to_vec(), true), + (earlier_node_id.to_vec(), false) + ] + ); + assert_eq!( + stored_operator_payout(&state, &payout_was_node_id), + Some(earlier_node_id) + ); +} + +/// A new operator whose payout Core reports equal to the platform node id gets an identity with +/// both keys: the payout's TRANSFER key and the platform node id key. +#[test] +fn should_create_a_new_operator_identity_whose_payout_equals_the_platform_node_id() { + let masternode = evonode(0x6d); + let node_id = masternode + .state + .platform_node_id + .expect("expected an evonode"); + let (platform, _state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![masternode.clone()], + vec![vec![( + masternode.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![0x7d; 48]), + operator_payout_address: Some(Some(node_id)), + ..empty_state_diff() + }, + )]], + ); + + assert_eq!( + keys_with_purpose(&platform, &masternode, &[0x7d; 48], Purpose::TRANSFER), + [(node_id.to_vec(), false)] + ); + assert_eq!( + keys_with_purpose(&platform, &masternode, &[0x7d; 48], Purpose::SYSTEM), + [(vec![0x7d; 48], false), (node_id.to_vec(), false)] + ); +} + +/// Core reports a masternode as updated on nearly every block for its payment fields alone; such +/// a diff leaves the stored list and the platform state clean, so the block does not rewrite the +/// saved state. +#[test] +fn should_leave_the_state_clean_after_a_payment_only_diff() { + let masternode = masternode_with_operator_payout(0x6e); + let (_platform, state) = masternode_list_after_diffs( + PlatformVersion::latest(), + vec![masternode.clone()], + vec![vec![( + masternode.pro_tx_hash, + DMNStateDiff { + last_paid_height: Some(1001), + consecutive_payments: Some(1), + ..empty_state_diff() + }, + )]], + ); + + assert!(!state.heavy_fields_dirty); + assert!(state.masternode_changes.upserted.is_empty()); +} + +/// Core empties the operator payout on every operator key change and prints nothing for it, so +/// a key change printed without a payout does not tell whether the new operator set a payout +/// again inside the same Core heights, possibly the very one the masternode had. Platform takes +/// the payout from Core's state of that masternode at the block's Core height instead: a new +/// operator identity gets the payout Core reports, an identity a masternode returns to keeps the +/// key of the payout Core reports, and a reset leaves no payout. Core is asked only about such +/// entries. +#[test] +fn should_take_the_payout_core_reports_for_an_operator_key_change_printed_without_one() { + let restored_on_new = masternode_with_operator_payout(0x71); + let restored_on_returning = masternode_with_operator_payout(0x72); + let reset = masternode_with_operator_payout(0x73); + let key_change = |masternode: &MasternodeListItem, key: u8, payout: Option| { + ( + masternode.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![key; 48]), + operator_payout_address: payout.map(|payout| Some([payout; 20])), + ..empty_state_diff() + }, + ) + }; + let (platform, state, reads) = masternode_list_after_diffs_with_core_payouts( + PlatformVersion::latest(), + vec![ + restored_on_new.clone(), + restored_on_returning.clone(), + reset.clone(), + ], + vec![ + vec![ + key_change(&restored_on_new, 0x81, None), + key_change(&restored_on_returning, 0x82, Some(0x92)), + key_change(&reset, 0x83, None), + ], + vec![key_change(&restored_on_returning, 0x72, None)], + ], + &[ + (restored_on_new.pro_tx_hash, 1001, Some([0x71; 20])), + (restored_on_returning.pro_tx_hash, 1002, Some([0x72; 20])), + ], + |_, _| {}, + ); + + assert_eq!( + transfer_keys(&platform, &restored_on_new, &[0x81; 48]), + [(vec![0x71; 20], false)] + ); + assert_eq!( + stored_operator_payout(&state, &restored_on_new), + Some([0x71; 20]) + ); + assert_eq!( + transfer_keys(&platform, &restored_on_returning, &[0x72; 48]), + [(vec![0x72; 20], false)] + ); + assert_eq!( + stored_operator_payout(&state, &restored_on_returning), + Some([0x72; 20]) + ); + assert_eq!(transfer_keys(&platform, &reset, &[0x83; 48]), []); + assert_eq!(stored_operator_payout(&state, &reset), None); + assert_eq!( + *reads.lock().unwrap(), + vec![ + (restored_on_new.pro_tx_hash, 1001), + (reset.pro_tx_hash, 1001), + (restored_on_returning.pro_tx_hash, 1002), + ] + ); +} + +/// The payout of an operator key change printed without one comes only from Core: a Core that +/// cannot answer fails the update, instead of the payout being read as cleared, and leaves the +/// stored list as it was. +#[test] +fn should_fail_when_core_cannot_report_the_payout_of_an_operator_key_change() { + let platform_version = PlatformVersion::latest(); + let masternode = masternode_with_operator_payout(0x75); + let mut platform = TestPlatformBuilder::new() + .with_config(PlatformConfig::default()) + .build_with_mock_rpc() + .set_genesis_state(); + let listed = MasternodeListDiff { + base_height: 0, + block_height: LIST_HEIGHT, + added_mns: vec![masternode.clone()], + removed_mns: vec![], + updated_mns: vec![], + }; + let key_change = MasternodeListDiff { + base_height: LIST_HEIGHT, + block_height: LIST_HEIGHT + 1, + added_mns: vec![], + removed_mns: vec![], + updated_mns: vec![( + masternode.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![0x85; 48]), + ..empty_state_diff() + }, + )], + }; + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |_, block| { + Ok(if block == LIST_HEIGHT { + listed.clone() + } else { + key_change.clone() + }) + }); + platform + .core_rpc + .expect_get_masternode_state() + .returning(|_, _| Err(CoreRpcError::UnexpectedStructure("refused".to_string()))); + + let transaction = platform.drive.grove.start_transaction(); + let mut state = platform.state.load().as_ref().clone(); + let block_info = |height, core_height| BlockInfo { + height, + core_height, + time_ms: height, + ..Default::default() + }; + platform + .update_masternode_list( + None, + &mut state, + LIST_HEIGHT, + true, + &block_info(1, LIST_HEIGHT), + &transaction, + platform_version, + ) + .expect("expected to list the masternode"); + let committed = state.clone(); + + let result = platform.update_masternode_list( + Some(&committed), + &mut state, + LIST_HEIGHT + 1, + false, + &block_info(2, LIST_HEIGHT + 1), + &transaction, + platform_version, + ); + + assert!(matches!(result, Err(Error::CoreRpc(_))), "{result:?}"); + assert_eq!( + state.full_masternode_list(), + committed.full_masternode_list() + ); +} + +/// An entry for a masternode Platform does not list is left as Core reports it, without asking +/// Core for the masternode's state: the list update applies nothing for it. +#[test] +fn should_not_ask_core_about_a_masternode_platform_does_not_list() { + let platform_version = PlatformVersion::latest(); + let listed = masternode_with_operator_payout(0x76); + let unlisted = masternode_with_operator_payout(0x77); + let mut platform = TestPlatformBuilder::new().build_with_mock_rpc(); + // Without the default answers, so asking Core anything else fails the test. + platform.core_rpc.checkpoint(); + let entry = ( + unlisted.pro_tx_hash, + DMNStateDiff { + pub_key_operator: Some(vec![0x87; 48]), + ..empty_state_diff() + }, + ); + let reported = entry.clone(); + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |base_block, block| { + Ok(MasternodeListDiff { + base_height: base_block.unwrap_or_default(), + block_height: block, + added_mns: vec![], + removed_mns: vec![], + updated_mns: vec![reported.clone()], + }) + }); + let mut state = PlatformState::default_with_protocol_versions( + platform_version.protocol_version, + platform_version.protocol_version, + &PlatformConfig::default(), + ) + .expect("expected a platform state"); + state.insert_masternode(listed); + let listed_before = state.full_masternode_list().clone(); + let transaction = platform.drive.grove.start_transaction(); + + let outcome = platform + .update_state_masternode_list_v2( + &mut state, + LIST_HEIGHT + 1, + false, + &transaction, + platform_version, + ) + .expect("expected to update the masternode list"); + + assert_eq!(outcome.masternode_list_diff.updated_mns, vec![entry]); + assert_eq!(state.full_masternode_list(), &listed_before); +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/mod.rs index 4024e7420de..9594ba98a77 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/mod.rs @@ -1,8 +1,10 @@ mod v0; mod v1; +mod v2; // Private helpers are selected by the update_masternode_list generation, except -// `without_operator_payout_changes` and `drop_unchanged_operator_keys`, which v0 and v1 share. +// `without_operator_payout_changes` and `drop_unchanged_operator_keys`, which v0 and v1 share, +// and `is_basic_re_encoding_of_legacy_key`, which v0, v1 and v2 share. use crate::error::execution::ExecutionError; use crate::error::Error; @@ -38,7 +40,8 @@ const BASIC_SIGN_BIT: u8 = 0x20; /// Whether `new` is the basic-scheme encoding of the BLS public key that `old` encodes in the /// legacy scheme. The verdict decides whether a masternode keeps its operator identity, so the -/// encodings it accepts are pinned by tests. +/// encodings it accepts are pinned by tests. `update_state_masternode_list` v0, v1 and v2 use it, +/// so a change of behaviour here needs new generations of all three. /// /// The legacy flags are moved to their basic-scheme positions and the result must decode as a /// basic-scheme key: an x coordinate below the field modulus, a point on the curve and in the diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v1/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v1/mod.rs index eded5d1e656..fce246ac6ba 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v1/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v1/mod.rs @@ -393,8 +393,10 @@ mod tests { fn state_with_one_masternode_and_diff( state_diff: DMNStateDiff, ) -> (TempPlatform, PlatformState, ProTxHash) { - let platform_version = PlatformVersion::latest(); - let mut platform = TestPlatformBuilder::new().build_with_mock_rpc(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) + .build_with_mock_rpc(); let pro_tx_hash = ProTxHash::from_byte_array([0x77u8; 32]); let mut state = PlatformState::default_with_protocol_versions( @@ -509,8 +511,10 @@ mod tests { #[test] fn should_only_upsert_changed_entries_in_a_mixed_masternode_diff() { - let platform_version = PlatformVersion::latest(); - let mut platform = TestPlatformBuilder::new().build_with_mock_rpc(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) + .build_with_mock_rpc(); let mut state = PlatformState::default_with_protocol_versions( platform_version.protocol_version, platform_version.protocol_version, @@ -637,8 +641,10 @@ mod tests { mut evonode: MasternodeListItem, diffs: Vec, ) -> PlatformState { - let platform_version = PlatformVersion::latest(); - let mut platform = TestPlatformBuilder::new().build_with_mock_rpc(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) + .build_with_mock_rpc(); let pro_tx_hash = evonode.pro_tx_hash; let quorum_hash = QuorumHash::from_byte_array(VALIDATOR_QUORUM_HASH); @@ -748,8 +754,9 @@ mod tests { /// `addresses`. #[test] fn should_hold_after_every_update_the_masternode_lists_a_restarted_node_loads() { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(14) .with_config(PlatformConfig { testing_configs: PlatformTestConfig { store_platform_state: true, @@ -1194,7 +1201,7 @@ mod port_resolution_tests { } fn validator_for(dmn_state: DMNState) -> Option { - let platform_version = PlatformVersion::latest(); + let platform_version = PlatformVersion::get(14).expect("expected protocol version 14"); let mut state = PlatformState::default_with_protocol_versions( platform_version.protocol_version, platform_version.protocol_version, diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v2/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v2/mod.rs new file mode 100644 index 00000000000..a3e0cc7a5a0 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/update_state_masternode_list/v2/mod.rs @@ -0,0 +1,393 @@ +use super::is_basic_re_encoding_of_legacy_key; +use crate::error::Error; +use crate::execution::types::update_state_masternode_list_outcome; +use crate::platform_types::masternode::v0::MasternodeStateV0; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::platform_types::platform_state::PlatformStateV0Methods; + +use crate::platform_types::validator_set::v0::ValidatorSetV0Getters; +use crate::rpc::core::CoreRPCLike; +use dpp::dashcore::{ProTxHash, QuorumHash}; +use dpp::dashcore_rpc::dashcore_rpc_json::{ + DMNState, DMNStateDiff, MasternodeListDiff, MasternodeListItem, +}; +use dpp::version::PlatformVersion; +use drive::drive::system::core_fork_activations::CoreFork; +use drive::grovedb::Transaction; +use std::collections::{BTreeMap, BTreeSet}; + +impl Platform +where + C: CoreRPCLike, +{ + /// The quorum hashes of the validator sets the masternode is a member of. + fn validator_sets_with_member_v2( + state: &PlatformState, + pro_tx_hash: &ProTxHash, + ) -> Vec { + state + .validator_sets() + .iter() + .filter(|(_, validator_set)| validator_set.members().contains_key(pro_tx_hash)) + .map(|(quorum_hash, _)| *quorum_hash) + .collect() + } + + /// Remove a masternode from every validator set it is a member of. + /// + /// Only the validator sets that list the masternode are touched, so only + /// their entries are rewritten by the per-entry store. + fn remove_masternode_in_validator_sets_v2(pro_tx_hash: &ProTxHash, state: &mut PlatformState) { + for quorum_hash in Self::validator_sets_with_member_v2(state, pro_tx_hash) { + if let Some(validator_set) = state.validator_set_mut(&quorum_hash) { + validator_set.members_mut().remove(pro_tx_hash); + } + } + } + + /// Updates a masternode in the validator sets. + /// + /// This function updates the properties of the masternode that matches the given `pro_tx_hash`. + /// The ban status and service address are updated based on the provided `dmn_state_diff` + /// information, the platform P2P and HTTP ports from `new_platform_ports`. + /// + /// # Arguments + /// + /// * `pro_tx_hash` - The `ProTxHash` of the masternode to be updated + /// * `dmn_state_diff` - The `DMNStateDiff` containing the updated masternode information + /// * `new_platform_ports` - The platform ports that resolve to a new value in the updated + /// masternode state + /// * `state` - The platform state whose validator sets are updated; only the + /// sets that list the masternode are touched + fn update_masternode_in_validator_sets_v2( + pro_tx_hash: &ProTxHash, + dmn_state_diff: &DMNStateDiff, + new_platform_ports: PlatformPorts, + state: &mut PlatformState, + ) { + for quorum_hash in Self::validator_sets_with_member_v2(state, pro_tx_hash) { + let Some(validator_set) = state.validator_set_mut(&quorum_hash) else { + continue; + }; + let Some(validator) = validator_set.members_mut().get_mut(pro_tx_hash) else { + continue; + }; + if let Some(maybe_ban_height) = dmn_state_diff.pose_ban_height { + // the ban_height was changed + validator.is_banned = maybe_ban_height.is_some(); + } + if let Some(address) = dmn_state_diff.service { + validator.node_ip = address.ip().to_string(); + } + + if let Some(p2p_port) = new_platform_ports.p2p { + validator.platform_p2p_port = p2p_port; + } + + if let Some(http_port) = new_platform_ports.http { + validator.platform_http_port = http_port; + } + } + } + + /// Whether applying `updated_mns` would change any item of `masternode_list`. + /// + /// Core reports a masternode as updated whenever any field of its state + /// moved, including the payment and penalty fields — last paid height, + /// consecutive payments, PoSe penalty — that change on nearly every Core + /// block and that the stored item does not keep. Applying each diff to a + /// copy tells the two apart. + fn any_stored_masternode_changes_v2( + masternode_list: &BTreeMap, + updated_mns: &[(ProTxHash, DMNStateDiff)], + ) -> bool { + updated_mns.iter().any(|(pro_tx_hash, state_diff)| { + masternode_list.get(pro_tx_hash).is_some_and(|item| { + apply_diff_in_stored_form(&item.state, state_diff) + != legacy_stored_form(item.state.clone()) + }) + }) + } + + /// Clears `pub_key_operator` in each entry of `updated_mns` whose reported key is the key + /// `masternode_list` already holds for that masternode, once Dash Core's V24 is recorded as + /// active: the same bytes, or the basic-scheme encoding of the key the stored bytes encode in + /// the legacy scheme. + /// + /// Once V24 is active, Core re-serializes a legacy-scheme operator key in the basic scheme + /// when it raises a masternode's state version, and reports the new bytes as a changed key. + /// The key is the same, so the masternode keeps the operator identity derived from the + /// stored bytes. Before V24 every entry stays as Core reports it. + /// + /// V24 is read from the Core fork record, at most once and only when an entry would be + /// cleared. This runs inside `update_core_info`, after it recorded the forks active at + /// `core_block_height` and only when that height moved, so the record answers for this very + /// height, as Core would. + fn drop_unchanged_operator_keys_v2( + &self, + masternode_list: &BTreeMap, + updated_mns: &mut [(ProTxHash, DMNStateDiff)], + transaction: &Transaction, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + let mut v24_active = false; + for (pro_tx_hash, state_diff) in updated_mns.iter_mut() { + let (Some(masternode), Some(new_key)) = ( + masternode_list.get(pro_tx_hash), + state_diff.pub_key_operator.as_deref(), + ) else { + continue; + }; + let stored_key = masternode.state.pub_key_operator.as_slice(); + if new_key != stored_key && !is_basic_re_encoding_of_legacy_key(stored_key, new_key) { + continue; + } + if !v24_active { + if self + .drive + .fetch_core_fork_activation_height( + CoreFork::V24, + Some(transaction), + platform_version, + )? + .is_none() + { + return Ok(()); + } + v24_active = true; + } + state_diff.pub_key_operator = None; + } + Ok(()) + } + + /// Updates the masternode lists like version 1, except for operator payouts and revived + /// heights. + /// + /// A diff's operator payout reaches the stored list and the identity updates. Core empties + /// the payout on every operator key change and prints no field for that, and it prints a + /// payout only when it differs between the diff's two Core heights, so a key change printed + /// without a payout does not tell whether the new operator set one again in between, maybe + /// the very payout the masternode had. For such an entry the payout is Core's for that + /// masternode at `core_block_height` (`get_masternode_state`, `protx info` at that chain + /// locked height), `Some(None)` when it has none: the new operator never starts with the + /// previous operator's payout, nor without its own. Core is asked about no other entry, and + /// not about a masternode Platform does not list, whose entry applies nothing; a Core error + /// fails the update. A diff applies with `DMNState::apply_diff`, which keeps a stored revived + /// height the diff does not print. + /// + /// An operator key Core only re-encoded is the stored key, and is dropped from its entry + /// first, so a re-encode is never read as a key change. + pub(crate) fn update_state_masternode_list_v2( + &self, + state: &mut PlatformState, + core_block_height: u32, + start_from_scratch: bool, + transaction: &Transaction, + platform_version: &PlatformVersion, + ) -> Result + { + let previous_core_height = if start_from_scratch { + // baseBlock must be a chain height and not 0 + None + } else { + let state_core_height = state.last_committed_core_height(); + if core_block_height == state_core_height { + return Ok(update_state_masternode_list_outcome::v0::UpdateStateMasternodeListOutcome::default()); + // no need to do anything + } + Some(state_core_height) + }; + + let mut masternode_diff = self + .core_rpc + .get_protx_diff_with_masternodes(previous_core_height, core_block_height)?; + + if !start_from_scratch { + self.drop_unchanged_operator_keys_v2( + state.full_masternode_list(), + &mut masternode_diff.updated_mns, + transaction, + platform_version, + )?; + for (pro_tx_hash, state_diff) in &mut masternode_diff.updated_mns { + if state_diff.pub_key_operator.is_none() + || state_diff.operator_payout_address.is_some() + || !state.full_masternode_list().contains_key(pro_tx_hash) + { + continue; + } + let core_state = self + .core_rpc + .get_masternode_state(pro_tx_hash, core_block_height) + .inspect_err(|error| { + tracing::error!( + %pro_tx_hash, + core_block_height, + ?error, + "could not read the operator payout of a masternode whose operator key changed" + ) + })?; + state_diff.operator_payout_address = Some(core_state.operator_payout_address); + } + } + + let MasternodeListDiff { + added_mns, + removed_mns, + updated_mns, + .. + } = &masternode_diff; + + // Core advances a block without any stored masternode field changing far + // more often than not: on mainnet almost every block's diff is only the + // payment fields of the masternode paid in it. Returning before the + // first mutable borrow keeps the platform state clean, which is what + // lets the block skip rewriting the full saved state (over a megabyte + // on mainnet) to disk. + if !start_from_scratch + && added_mns.is_empty() + && removed_mns.is_empty() + && !Self::any_stored_masternode_changes_v2(state.full_masternode_list(), updated_mns) + { + return Ok( + update_state_masternode_list_outcome::v0::UpdateStateMasternodeListOutcome { + masternode_list_diff: masternode_diff, + removed_masternodes: BTreeMap::new(), + }, + ); + } + + // Every change below goes through an accessor that records which + // masternode or validator set it touched, so the store writes only + // those entries. + if start_from_scratch { + state.clear_masternode_lists(); + } + + for masternode in added_mns { + let mut masternode = masternode.clone(); + masternode.state = in_stored_form(masternode.state); + state.insert_masternode(masternode); + } + + for (pro_tx_hash, state_diff) in updated_mns { + let Some(mut masternode) = state.full_masternode_list().get(pro_tx_hash).cloned() + else { + continue; + }; + // Old in-memory addresses were not persisted before version 1. Start + // from the stored state so a running node and a restarted node agree. + let previous_state = legacy_stored_form(masternode.state.clone()); + let updated_state = apply_diff_in_stored_form(&previous_state, state_diff); + let is_hpmn = state.hpmn_masternode_list().contains_key(pro_tx_hash); + let platform_ports_before = is_hpmn.then(|| PlatformPorts::of(&previous_state)); + let platform_ports = is_hpmn.then(|| PlatformPorts::of(&updated_state)); + if updated_state != previous_state { + masternode.state = updated_state; + state.insert_masternode(masternode); + } + // Only evonodes are validators. + let (Some(platform_ports_before), Some(platform_ports)) = + (platform_ports_before, platform_ports) + else { + continue; + }; + // The ports are read from the updated state, not from the diff: an + // ExtAddr evonode's port move reaches us only in `addresses`, and a + // legacy evonode's diff lists there just the entries that changed. + let new_platform_ports = changed_platform_ports(platform_ports_before, platform_ports); + // the ban status, the IP and the platform P2P port are the only fields that are useful for + // validators. If they change we need to update validator sets + if state_diff.pose_ban_height.is_some() + || state_diff.service.is_some() + || new_platform_ports.p2p.is_some() + { + // we updated the ban status the IP or the platform port, we need to update the validator in the validator list + Self::update_masternode_in_validator_sets_v2( + pro_tx_hash, + state_diff, + new_platform_ports, + state, + ); + } + } + + for pro_tx_hash in removed_mns { + Self::remove_masternode_in_validator_sets_v2(pro_tx_hash, state); + } + + let deleted_masternodes = removed_mns.iter().copied().collect::>(); + + let mut removed_masternodes = BTreeMap::new(); + + for key in deleted_masternodes { + if let Some(value) = state.remove_masternode(&key) { + removed_masternodes.insert(key, value); + } + } + + Ok( + update_state_masternode_list_outcome::v0::UpdateStateMasternodeListOutcome { + masternode_list_diff: masternode_diff, + removed_masternodes, + }, + ) + } +} + +/// The ports of `after` that resolve to a value other than in `before`. +/// +/// A port that stops resolving is not reported, so the validator keeps its +/// last known port. +fn changed_platform_ports(before: PlatformPorts, after: PlatformPorts) -> PlatformPorts { + PlatformPorts { + p2p: after.p2p.filter(|_| after.p2p != before.p2p), + http: after.http.filter(|_| after.http != before.http), + } +} + +/// The old persisted state, without transient address or payout lists. +fn legacy_stored_form(state: DMNState) -> DMNState { + MasternodeStateV0::from(state).into() +} + +/// Resolves nested ports into the unchanged persisted fields. The global +/// converters stay literal so earlier protocol generations keep their behavior. +fn in_stored_form(mut state: DMNState) -> DMNState { + let p2p = state.platform_p2p_address().map(|(_, port)| port); + let http = state.platform_http_address().map(|(_, port)| port); + #[allow(deprecated)] + { + state.legacy_platform_p2p_port = p2p.or(state.legacy_platform_p2p_port); + state.legacy_platform_http_port = http.or(state.legacy_platform_http_port); + } + legacy_stored_form(state) +} + +fn apply_diff_in_stored_form(state: &DMNState, diff: &DMNStateDiff) -> DMNState { + let mut updated = legacy_stored_form(state.clone()); + updated.apply_diff(diff.clone()); + in_stored_form(updated) +} + +/// Platform ports resolved for this generation's validator updates. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct PlatformPorts { + p2p: Option, + http: Option, +} + +impl PlatformPorts { + fn of(state: &DMNState) -> Self { + Self { + p2p: state + .platform_p2p_address() + .and_then(|(_, port)| u16::try_from(port).ok()), + http: state + .platform_http_address() + .and_then(|(_, port)| u16::try_from(port).ok()), + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/v2/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/v2/mod.rs new file mode 100644 index 00000000000..0a32f081c7a --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/v2/mod.rs @@ -0,0 +1,100 @@ +use crate::error::Error; +use crate::execution::types::update_state_masternode_list_outcome; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::platform_types::platform_state::PlatformStateV0Methods; +use crate::rpc::core::CoreRPCLike; +use dpp::block::block_info::BlockInfo; +use dpp::block::extended_block_info::v0::ExtendedBlockInfoV0Getters; +use dpp::version::PlatformVersion; +use drive::grovedb::Transaction; + +impl Platform +where + C: CoreRPCLike, +{ + /// Updates the masternode list in the platform state based on changes in the masternode list + /// from Dash Core between two block heights. + /// + /// Version 1's steps with `update_state_masternode_list_v2`, which keeps a diff's operator + /// payouts, reads from Core the payout of an operator key change printed without one, keeps + /// revived heights a diff does not print, and reads whether Dash Core's V24 is active from + /// the Core fork record. + /// + /// # Arguments + /// + /// * `platform_state` - The optional committed platform state used for identity updates. + /// * `block_platform_state` - The mutable platform state for the block being executed. + /// * `core_block_height` - The current block height in the Dash Core. + /// * `is_init_chain` - Whether the update is initializing the chain. + /// * `block_info` - The block information used for identity updates. + /// * `transaction` - The current groveDB transaction. + /// * `platform_version` - The protocol version governing the update. + /// + /// # Returns + /// + /// * `Result<(), Error>` - Returns `Ok(())` if the update is successful. Returns an error if + /// there is a problem fetching the masternode list difference or updating the state. + #[allow(clippy::too_many_arguments)] + pub(super) fn update_masternode_list_v2( + &self, + platform_state: Option<&PlatformState>, + block_platform_state: &mut PlatformState, + core_block_height: u32, + is_init_chain: bool, + block_info: &BlockInfo, + transaction: &Transaction, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + if let Some(last_committed_block_info) = + block_platform_state.last_committed_block_info().as_ref() + { + if core_block_height == last_committed_block_info.basic_info().core_height { + tracing::debug!( + method = "update_masternode_list_v2", + "no update mnl at height {}", + core_block_height, + ); + return Ok(()); // no need to do anything + } + } + tracing::debug!( + method = "update_masternode_list_v2", + "update mnl to height {} at block {}", + core_block_height, + block_platform_state.last_committed_core_height() + ); + + let update_state_masternode_list_outcome::v0::UpdateStateMasternodeListOutcome { + masternode_list_diff, + removed_masternodes, + } = self.update_state_masternode_list_v2( + block_platform_state, + core_block_height, + is_init_chain, + transaction, + platform_version, + )?; + + self.update_masternode_identities( + masternode_list_diff, + &removed_masternodes, + block_info, + platform_state, + transaction, + platform_version, + )?; + + if !removed_masternodes.is_empty() { + self.drive.remove_validators_proposed_app_versions( + removed_masternodes + .into_keys() + .map(|pro_tx_hash| pro_tx_hash.into()), + Some(transaction), + &platform_version.drive, + )?; + } + + Ok(()) + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/versioning_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/versioning_tests.rs index 345db4744cd..648354d6cea 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/versioning_tests.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/core_based_updates/update_masternode_list/versioning_tests.rs @@ -168,3 +168,60 @@ fn should_use_nested_ports_only_after_the_protocol_upgrade() { ); } } + +/// Historical freeze: protocol versions 1 to 14 executed their blocks with +/// `update_masternode_list` 0 or 1 and `update_operator_identity` 0, which ignore the operator +/// payouts of Core's diffs, so they must keep selecting them for those blocks to replay to the +/// same state. The generations that apply the payouts are exercised through the dispatcher at +/// the latest protocol version (`operator_payout_tests`). +#[test] +fn should_keep_the_masternode_generations_protocol_versions_1_to_14_executed_with() { + for protocol_version in 1..=14 { + let core_based_updates = &PlatformVersion::get(protocol_version) + .expect("expected a protocol version") + .drive_abci + .methods + .core_based_updates; + let update_masternode_list = if protocol_version == 14 { 1 } else { 0 }; + assert_eq!( + ( + core_based_updates.update_masternode_list, + core_based_updates + .masternode_updates + .update_operator_identity + ), + (update_masternode_list, 0), + "protocol version {protocol_version}" + ); + } +} + +/// `update_operator_identity` 1 adds a key whose hash another key of the identity has, which +/// Drive's `add_new_keys_to_masternode_identity` 0 refuses on every node, failing the block. So +/// every protocol version selecting the first selects version 1 or later of the second. +#[test] +fn should_pair_update_operator_identity_1_with_a_drive_that_adds_keys_repeating_a_hash() { + for protocol_version in 1..=PlatformVersion::latest().protocol_version { + let platform_version = + PlatformVersion::get(protocol_version).expect("expected a protocol version"); + if platform_version + .drive_abci + .methods + .core_based_updates + .masternode_updates + .update_operator_identity + >= 1 + { + assert!( + platform_version + .drive + .methods + .identity + .update + .add_new_keys_to_masternode_identity + >= 1, + "protocol version {protocol_version}" + ); + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/credit_pool_backfill_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/credit_pool_backfill_tests.rs index 59a9754b2d4..70c5cf582cd 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/credit_pool_backfill_tests.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/credit_pool_backfill_tests.rs @@ -1,3 +1,4 @@ +use crate::platform_types::platform_state::PlatformState; use crate::test::helpers::setup::TestPlatformBuilder; use dpp::block::block_info::BlockInfo; use dpp::version::PlatformVersion; @@ -12,6 +13,7 @@ fn should_backfill_credit_notes_when_skipping_versions_and_leave_pv13_replay_unc let platform = TestPlatformBuilder::new() .with_initial_protocol_version(previous) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let rho = [7; 32]; if previous >= 12 { @@ -38,6 +40,7 @@ fn should_backfill_credit_notes_when_skipping_versions_and_leave_pv13_replay_unc platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &BlockInfo::default(), &tx, previous, @@ -55,6 +58,7 @@ fn should_backfill_credit_notes_when_skipping_versions_and_leave_pv13_replay_unc platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &BlockInfo::default(), &tx, previous, @@ -83,6 +87,7 @@ fn should_backfill_credit_notes_when_skipping_versions_and_leave_pv13_replay_unc platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &BlockInfo::default(), &tx, latest.protocol_version, @@ -107,6 +112,7 @@ fn should_keep_fresh_pv14_genesis_empty_when_backfill_is_repeated() { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(pv.protocol_version) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let original = platform .drive diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/mod.rs index fb8bc3b0e3d..083dd0dc433 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/mod.rs @@ -5,18 +5,24 @@ mod v2; #[cfg(test)] mod credit_pool_backfill_tests; #[cfg(test)] +mod operator_payout_reconciliation_tests; +#[cfg(test)] mod token_history_storage_flags_tests; use crate::error::execution::ExecutionError; use crate::error::Error; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::PlatformState; +use crate::rpc::core::CoreRPCLike; use dpp::block::block_info::BlockInfo; use dpp::version::PlatformVersion; use dpp::version::ProtocolVersion; use drive::grovedb::Transaction; -impl Platform { +impl Platform +where + C: CoreRPCLike, +{ /// Executes protocol-specific events on the first block after a protocol version change. /// /// This function is triggered when there is a protocol version upgrade detected in the network. @@ -28,6 +34,9 @@ impl Platform { /// /// # Parameters /// + /// * `platform_state`: The committed platform state, still at the previous protocol version. + /// * `block_platform_state`: The platform state of the block being executed, for events that + /// repair Platform's copy of Dash Core's masternode list. /// * `transaction`: A reference to the transaction context in which the changes should be applied. /// * `previous_protocol_version`: The protocol version prior to the upgrade. /// * `platform_version`: The current platform version containing the updated protocol version and relevant configuration details. @@ -55,6 +64,7 @@ impl Platform { pub fn perform_events_on_first_block_of_protocol_change( &self, platform_state: &PlatformState, + block_platform_state: &mut PlatformState, block_info: &BlockInfo, transaction: &Transaction, previous_protocol_version: ProtocolVersion, @@ -68,6 +78,7 @@ impl Platform { { Some(0) => self.perform_events_on_first_block_of_protocol_change_v0( platform_state, + block_platform_state, block_info, transaction, previous_protocol_version, @@ -75,6 +86,7 @@ impl Platform { ), Some(1) => self.perform_events_on_first_block_of_protocol_change_v1( platform_state, + block_platform_state, block_info, transaction, previous_protocol_version, @@ -82,6 +94,7 @@ impl Platform { ), Some(2) => self.perform_events_on_first_block_of_protocol_change_v2( platform_state, + block_platform_state, block_info, transaction, previous_protocol_version, @@ -117,6 +130,7 @@ mod tests { let platform_version = PlatformVersion::latest(); let platform = TestPlatformBuilder::new() .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let transaction = platform.drive.grove.start_transaction(); @@ -134,6 +148,7 @@ mod tests { // since all transition conditions (previous < X && current >= X) will be false let result = platform.perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, platform_version.protocol_version, @@ -148,6 +163,7 @@ mod tests { let platform_version = PlatformVersion::latest(); let platform = TestPlatformBuilder::new() .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let transaction = platform.drive.grove.start_transaction(); @@ -170,6 +186,7 @@ mod tests { let result = platform.perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, platform_version.protocol_version, @@ -198,6 +215,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(13) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); let block_info = BlockInfo { @@ -247,6 +265,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, previous_version.protocol_version, @@ -277,6 +296,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(13) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); let block_info = BlockInfo { @@ -301,6 +321,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, 13, @@ -390,6 +411,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); let block_info = BlockInfo { @@ -414,6 +436,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, previous_version.protocol_version, diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/operator_payout_reconciliation_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/operator_payout_reconciliation_tests.rs new file mode 100644 index 00000000000..477dff250e5 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/operator_payout_reconciliation_tests.rs @@ -0,0 +1,710 @@ +//! The first block of protocol version 15 reconciles the operator payouts Platform holds with +//! Dash Core's masternode list, at the last committed chain locked core height. + +use crate::config::PlatformConfig; +use crate::platform_types::platform_state::{PlatformState, PlatformStateV0Methods}; +use crate::rpc::core::MockCoreRPCLike; +use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; +use dpp::block::block_info::BlockInfo; +use dpp::block::extended_block_info::v0::ExtendedBlockInfoV0; +use dpp::block::extended_block_info::ExtendedBlockInfo; +use dpp::dashcore::hashes::Hash; +use dpp::dashcore::{ProTxHash, Txid}; +use dpp::dashcore_rpc::dashcore_rpc_json::{ + DMNState, DMNStateDiff, ExtendedQuorumListResult, MasternodeListDiff, MasternodeListItem, + MasternodeType, +}; +use dpp::identifier::MasternodeIdentifiers; +use dpp::identity::accessors::IdentityGettersV0; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::Purpose; +use dpp::prelude::Identifier; +use dpp::version::{PlatformVersion, ProtocolVersion}; +use drive::grovedb::Transaction; +use std::collections::HashMap; +use std::net::SocketAddr; +use std::sync::{Arc, Mutex}; + +/// The Core height the masternodes are first listed at, and the last committed one, which the +/// upgrade block reconciles at. +const LISTED_AT: u32 = 999; +const COMMITTED_AT: u32 = 1000; +/// The core height the upgrade block moves to. +const UPGRADE_AT: u32 = 1001; + +/// A Regular masternode named by `id` with an operator reward, operator key `[key; 48]` and +/// `operator_payout`. +fn masternode(id: u8, key: u8, operator_payout: Option) -> MasternodeListItem { + MasternodeListItem { + node_type: MasternodeType::Regular, + pro_tx_hash: ProTxHash::from_byte_array([id; 32]), + collateral_hash: Txid::from_byte_array([id ^ 0x40; 32]), + collateral_index: 0, + collateral_address: Some([id ^ 0x50; 20]), + operator_reward: 10.0, + state: DMNState { + service: SocketAddr::from(([192, 0, 2, id], 9999)), + registered_height: 900, + pose_revived_height: None, + pose_ban_height: None, + revocation_reason: 0, + owner_address: Some([id ^ 0x10; 20]), + voting_address: [id ^ 0x20; 20], + payout_address: Some([id ^ 0x30; 20]), + payouts: None, + pub_key_operator: vec![key; 48], + operator_payout_address: operator_payout.map(|payout| [payout; 20]), + platform_node_id: None, + #[allow(deprecated)] + legacy_platform_p2p_port: None, + #[allow(deprecated)] + legacy_platform_http_port: None, + addresses: None, + }, + } +} + +fn diff_entry(fields: serde_json::Value) -> DMNStateDiff { + serde_json::from_value(fields).expect("expected a Core state diff") +} + +/// The masternode list calls Platform made to Core, as (base height, height). +type CoreCalls = Arc, u32)>>>; + +/// Three masternodes, as Core lists them at `LISTED_AT` and at `COMMITTED_AT`, and the diff Core +/// prints between the two: 0x61's operator sets another payout in place of 0x61, and 0x62's +/// owner sets operator key 0x72, which empties the payout 0x62. 0x64 does not change. +fn core_history() -> ( + Vec, + MasternodeListDiff, + Vec, +) { + let listed = vec![ + masternode(0x61, 0x61, Some(0x61)), + masternode(0x62, 0x62, Some(0x62)), + masternode(0x64, 0x64, Some(0x64)), + ]; + let diff = MasternodeListDiff { + base_height: LISTED_AT, + block_height: COMMITTED_AT, + added_mns: vec![], + removed_mns: vec![], + updated_mns: vec![ + ( + listed[0].pro_tx_hash, + diff_entry(serde_json::json!({ + "operatorPayoutAddress": "yXkZa7p2ne58x2XQYdNthAxfXy9cHiWxF5", + })), + ), + ( + listed[1].pro_tx_hash, + diff_entry(serde_json::json!({ "pubKeyOperator": "72".repeat(48) })), + ), + ], + }; + let mut at_committed = listed.clone(); + at_committed[0].state.operator_payout_address = diff.updated_mns[0] + .1 + .operator_payout_address + .expect("expected the diff to set the payout"); + at_committed[1].state.pub_key_operator = vec![0x72; 48]; + at_committed[1].state.operator_payout_address = None; + (listed, diff, at_committed) +} + +/// The diff of the upgrade block itself, `COMMITTED_AT` to `UPGRADE_AT`: 0x61's owner sets +/// operator key 0x71, 0x62's new operator sets a payout, 0x64 is removed and 0x65 registered. +fn upgrade_block_diff() -> MasternodeListDiff { + MasternodeListDiff { + base_height: COMMITTED_AT, + block_height: UPGRADE_AT, + added_mns: vec![masternode(0x65, 0x65, Some(0x65))], + removed_mns: vec![ProTxHash::from_byte_array([0x64; 32])], + updated_mns: vec![ + ( + ProTxHash::from_byte_array([0x61; 32]), + diff_entry(serde_json::json!({ "pubKeyOperator": "71".repeat(48) })), + ), + ( + ProTxHash::from_byte_array([0x62; 32]), + diff_entry(serde_json::json!({ + "operatorPayoutAddress": "yd59MzNMSWf5LdNE9qxZGrcFeR2BzjuqM2", + })), + ), + ], + } +} + +/// Runs the protocol change events of the first block of protocol version 15 after `previous`, +/// in `transaction`, on a fresh copy of `committed` as `run_block_proposal` makes for each round. +fn run_upgrade_events( + platform: &TempPlatform, + committed: &PlatformState, + previous: ProtocolVersion, + transaction: &Transaction, +) -> PlatformState { + let platform_version = PlatformVersion::latest(); + let mut block_state = committed.clone(); + block_state.set_current_protocol_version_in_consensus(platform_version.protocol_version); + platform + .perform_events_on_first_block_of_protocol_change( + committed, + &mut block_state, + &BlockInfo { + height: 3, + core_height: UPGRADE_AT, + ..Default::default() + }, + transaction, + previous, + platform_version, + ) + .expect("expected the protocol change events to succeed"); + block_state +} + +/// A platform at `previous` that processed `core_history` as Platform did at that protocol +/// version, through `update_core_info`: the list at `LISTED_AT` at init chain, then the diff in a +/// block at `COMMITTED_AT`. +/// Returns the committed state and the masternode list calls Core gets after that block. +fn platform_after_core_history( + previous: ProtocolVersion, +) -> (TempPlatform, PlatformState, CoreCalls) { + let (listed, diff, at_committed) = core_history(); + platform_after_history(previous, listed, vec![diff], at_committed) +} + +/// A platform at `previous` that processed, through `update_core_info`, Core's list `listed` at +/// init chain and then each of `diffs` in a block of its own, one Core height each, the last at +/// `COMMITTED_AT`. Core lists `at_committed` at `COMMITTED_AT` and prints `upgrade_block_diff` +/// after it. Returns the committed state and the masternode list calls Core gets after that. +fn platform_after_history( + previous: ProtocolVersion, + listed: Vec, + diffs: Vec, + at_committed: Vec, +) -> (TempPlatform, PlatformState, CoreCalls) { + platform_with_config_after_history( + PlatformConfig::default(), + previous, + listed, + diffs, + at_committed, + ) +} + +/// `platform_after_history` with `config`. +fn platform_with_config_after_history( + config: PlatformConfig, + previous: ProtocolVersion, + listed: Vec, + diffs: Vec, + at_committed: Vec, +) -> (TempPlatform, PlatformState, CoreCalls) { + let previous_version = PlatformVersion::get(previous).expect("expected the protocol version"); + let mut platform = TestPlatformBuilder::new() + .with_config(config) + .with_initial_protocol_version(previous) + .build_with_mock_rpc() + .set_genesis_state(); + let listed_at = COMMITTED_AT - diffs.len() as u32; + let calls = CoreCalls::default(); + let recorded_calls = Arc::clone(&calls); + let history_done = Arc::new(Mutex::new(false)); + let done = Arc::clone(&history_done); + platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .returning(move |base_block, block| { + if *done.lock().expect("expected the history flag") { + recorded_calls + .lock() + .expect("expected the Core calls") + .push((base_block, block)); + } + let added_mns = match (base_block, block) { + (None, block) if block == listed_at => listed.clone(), + (None, COMMITTED_AT) => at_committed.clone(), + (Some(COMMITTED_AT), UPGRADE_AT) => return Ok(upgrade_block_diff()), + (Some(base), block) if base >= listed_at && block == base + 1 => { + return Ok(diffs[(base - listed_at) as usize].clone()) + } + other => panic!("unexpected masternode list call {other:?}"), + }; + Ok(MasternodeListDiff { + base_height: 1, + block_height: block, + added_mns, + removed_mns: vec![], + updated_mns: vec![], + }) + }); + + // Asked about a masternode whose operator key changed without a printed payout, Core reports + // none, as after a key change that set none. Up to protocol version 14 nothing asks. + let state_reads = Arc::new(Mutex::new(0)); + let recorded_state_reads = Arc::clone(&state_reads); + platform + .core_rpc + .expect_get_masternode_state() + .returning(move |_, _| { + *recorded_state_reads + .lock() + .expect("expected the state reads") += 1; + Ok(masternode(0, 0, None).state) + }); + let quorum_types = [ + platform.config.validator_set.quorum_type, + platform.config.chain_lock.quorum_type, + platform.config.instant_lock.quorum_type, + ]; + platform + .core_rpc + .expect_get_quorum_listextended() + .returning(move |_| { + Ok(ExtendedQuorumListResult { + quorums_by_type: quorum_types + .iter() + .map(|quorum_type| (*quorum_type, Default::default())) + .collect::>(), + }) + }); + + let transaction = platform.drive.grove.start_transaction(); + let mut state = platform.state.load().as_ref().clone(); + let mut committed: Option = None; + for (height, core_height) in (1..).zip(listed_at..=COMMITTED_AT) { + platform + .update_core_info( + committed.as_ref(), + &mut state, + core_height, + committed.is_none(), + &BlockInfo { + height, + core_height, + ..Default::default() + }, + &transaction, + previous_version, + ) + .expect("expected to process Core's masternode list"); + state.set_last_committed_block_info(Some(committed_block(height, core_height))); + committed = Some(state.clone()); + } + state.mark_saved(); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected to commit"); + *history_done.lock().expect("expected the history flag") = true; + assert_eq!( + *state_reads.lock().expect("expected the state reads"), + 0, + "protocol version {previous} asked Core for a masternode's state" + ); + (platform, state, calls) +} + +fn committed_block(height: u64, core_height: u32) -> ExtendedBlockInfo { + ExtendedBlockInfoV0 { + basic_info: BlockInfo { + height, + core_height, + ..Default::default() + }, + app_hash: [0; 32], + quorum_hash: [0; 32], + block_id_hash: [0; 32], + proposer_pro_tx_hash: [0; 32], + signature: [0; 96], + round: 0, + } + .into() +} + +/// The (data, disabled) of every TRANSFER key of the operator identity of the masternode named +/// by `id` with operator key `[key; 48]`. +fn transfer_keys( + platform: &TempPlatform, + transaction: &Transaction, + id: u8, + key: u8, +) -> Vec<(Vec, bool)> { + platform + .drive + .fetch_full_identity( + Identifier::create_operator_identifier(&[id; 32], &[key; 48]).to_buffer(), + Some(transaction), + PlatformVersion::latest(), + ) + .expect("expected to fetch the operator identity") + .expect("expected the operator identity") + .public_keys() + .values() + .filter(|key| key.purpose() == Purpose::TRANSFER) + .map(|key| (key.data().to_vec(), key.is_disabled())) + .collect() +} + +/// Up to protocol version 14 a diff's operator payout never reached Platform, and a new operator +/// identity took the stored payout, the previous operator's. The first block of protocol version +/// 15, also after a jump from 13, reads Core's list at the last committed core height and fixes +/// both, in the block's state and its transaction: the changed payout's key replaces the old one, +/// and the new operator's identity loses the inherited key, which the previous operator's +/// identity keeps. +#[test] +fn should_reconcile_operator_payouts_with_core_in_the_first_block_of_protocol_version_15() { + let platform_version = PlatformVersion::latest(); + for previous in [13, 14] { + let (platform, committed, calls) = platform_after_core_history(previous); + assert_eq!( + committed.full_masternode_list()[&ProTxHash::from_byte_array([0x61; 32])] + .state + .operator_payout_address, + Some([0x61; 20]), + "PV{previous} kept the listed payout" + ); + let transaction = platform.drive.grove.start_transaction(); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x72), + [(vec![0x62; 20], false)], + "PV{previous} gave the new operator the previous operator's payout key" + ); + let mut block_state = committed.clone(); + block_state.set_current_protocol_version_in_consensus(platform_version.protocol_version); + + platform + .perform_events_on_first_block_of_protocol_change( + &committed, + &mut block_state, + &BlockInfo { + height: 3, + core_height: COMMITTED_AT + 1, + ..Default::default() + }, + &transaction, + previous, + platform_version, + ) + .expect("expected the protocol change events to succeed"); + + let context = format!("from PV{previous}"); + assert_eq!( + *calls.lock().unwrap(), + vec![(None, COMMITTED_AT)], + "{context}" + ); + let stored_payout = |id: u8| { + block_state.full_masternode_list()[&ProTxHash::from_byte_array([id; 32])] + .state + .operator_payout_address + }; + let (_, _, at_committed) = core_history(); + assert_eq!( + stored_payout(0x61), + at_committed[0].state.operator_payout_address, + "{context}" + ); + assert_eq!(stored_payout(0x62), None, "{context}"); + let new_payout = at_committed[0] + .state + .operator_payout_address + .expect("expected Core's payout") + .to_vec(); + assert_eq!( + transfer_keys(&platform, &transaction, 0x61, 0x61), + [(vec![0x61; 20], true), (new_payout, false)], + "{context}" + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x72), + [(vec![0x62; 20], true)], + "{context}" + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x62), + [(vec![0x62; 20], false)], + "{context}: the previous operator's identity keeps its payout key" + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x64, 0x64), + [(vec![0x64; 20], false)], + "{context}: an identity that already matches Core is left as it is" + ); + assert_eq!(stored_payout(0x64), Some([0x64; 20]), "{context}"); + assert!( + !block_state + .masternode_changes + .upserted + .contains(&ProTxHash::from_byte_array([0x64; 32])), + "{context}: a masternode whose payout matches Core's is not rewritten" + ); + } +} + +/// Only the upgrade to protocol version 15 reconciles: the first block of 14 does not read Core's +/// masternode list, and leaves the payouts as they are. +#[test] +fn should_not_reconcile_operator_payouts_in_the_first_block_of_protocol_version_14() { + let (platform, committed, calls) = platform_after_core_history(13); + let transaction = platform.drive.grove.start_transaction(); + let mut block_state = committed.clone(); + block_state.set_current_protocol_version_in_consensus(14); + + platform + .perform_events_on_first_block_of_protocol_change( + &committed, + &mut block_state, + &BlockInfo { + height: 3, + core_height: COMMITTED_AT + 1, + ..Default::default() + }, + &transaction, + 13, + PlatformVersion::get(14).expect("expected protocol version 14"), + ) + .expect("expected the protocol change events to succeed"); + + assert_eq!(*calls.lock().unwrap(), vec![]); + assert_eq!( + block_state.full_masternode_list(), + committed.full_masternode_list() + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x72), + [(vec![0x62; 20], false)] + ); +} + +/// The upgrade block then applies its own diff from the reconciled list, under protocol version +/// 15's rules, with the committed state as the identity updates' base as in a real block: a key +/// change takes Core's payout (none here), a payout reaches the identity the reconciliation +/// fixed, under a fresh key id, and masternodes come and go as always. +#[test] +fn should_apply_the_upgrade_blocks_own_diff_after_the_reconciliation() { + let platform_version = PlatformVersion::latest(); + let (platform, committed, calls) = platform_after_core_history(14); + let transaction = platform.drive.grove.start_transaction(); + let mut block_state = run_upgrade_events(&platform, &committed, 14, &transaction); + + platform + .update_core_info( + Some(&committed), + &mut block_state, + UPGRADE_AT, + false, + &BlockInfo { + height: 3, + core_height: UPGRADE_AT, + ..Default::default() + }, + &transaction, + platform_version, + ) + .expect("expected to apply the upgrade block's diff"); + + assert_eq!( + *calls.lock().unwrap(), + vec![(None, COMMITTED_AT), (Some(COMMITTED_AT), UPGRADE_AT)] + ); + let stored = |id: u8| { + block_state + .full_masternode_list() + .get(&ProTxHash::from_byte_array([id; 32])) + .map(|masternode| { + ( + masternode.state.pub_key_operator.clone(), + masternode.state.operator_payout_address, + ) + }) + }; + assert_eq!(stored(0x61), Some((vec![0x71; 48], None))); + let upgrade_diff = upgrade_block_diff(); + let new_payout = upgrade_diff.updated_mns[1] + .1 + .operator_payout_address + .expect("expected the diff to set the payout") + .expect("expected a payout"); + assert_eq!(stored(0x62), Some((vec![0x72; 48], Some(new_payout)))); + assert_eq!(stored(0x64), None); + assert_eq!(stored(0x65), Some((vec![0x65; 48], Some([0x65; 20])))); + + assert_eq!(transfer_keys(&platform, &transaction, 0x61, 0x71), []); + let (_, _, at_committed) = core_history(); + let reconciled_payout = at_committed[0] + .state + .operator_payout_address + .expect("expected Core's payout") + .to_vec(); + assert_eq!( + transfer_keys(&platform, &transaction, 0x61, 0x61), + [(vec![0x61; 20], true), (reconciled_payout, false)], + "the identity 0x61 leaves keeps the reconciled payout key" + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x72), + [(vec![0x62; 20], true), (new_payout.to_vec(), false)] + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x65, 0x65), + [(vec![0x65; 20], false)] + ); +} + +/// A round of the upgrade block that fails is thrown away with its transaction and its copy of +/// the state. The next round reconciles again, from the same committed state and Core's same +/// list, to the same result: nothing from the failed round is kept or applied twice. +#[test] +fn should_reconcile_the_same_way_when_a_round_of_the_upgrade_block_is_retried() { + let (platform, committed, calls) = platform_after_core_history(14); + { + let failed_round = platform.drive.grove.start_transaction(); + run_upgrade_events(&platform, &committed, 14, &failed_round); + } + let transaction = platform.drive.grove.start_transaction(); + let block_state = run_upgrade_events(&platform, &committed, 14, &transaction); + + assert_eq!( + *calls.lock().unwrap(), + vec![(None, COMMITTED_AT), (None, COMMITTED_AT)] + ); + let (_, _, at_committed) = core_history(); + assert_eq!( + block_state.full_masternode_list()[&ProTxHash::from_byte_array([0x61; 32])] + .state + .operator_payout_address, + at_committed[0].state.operator_payout_address + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x61, 0x61), + [ + (vec![0x61; 20], true), + ( + at_committed[0] + .state + .operator_payout_address + .expect("expected Core's payout") + .to_vec(), + false + ) + ] + ); + assert_eq!( + transfer_keys(&platform, &transaction, 0x62, 0x72), + [(vec![0x62; 20], true)] + ); +} + +/// The enabled TRANSFER keys of a current operator identity are not always its stored payout: up +/// to protocol version 14 a platform node id matched keys by data alone, so an evonode whose node +/// id was set to its payout's bytes and then changed again had its payout key disabled, while the +/// stored payout and Core's stayed the same. The first block of protocol version 15 reconciles +/// every current operator identity with Core's payout, so that key is enabled again. +#[test] +fn should_re_enable_a_payout_key_an_earlier_node_id_change_disabled_in_the_first_block_of_protocol_version_15( +) { + let mut evonode = masternode(0x66, 0x66, Some(0x66)); + evonode.node_type = MasternodeType::Evo; + evonode.state.platform_node_id = Some([0x76; 20]); + #[allow(deprecated)] + { + evonode.state.legacy_platform_p2p_port = Some(26656); + evonode.state.legacy_platform_http_port = Some(443); + } + let node_id_change = |node_id: [u8; 20]| MasternodeListDiff { + base_height: 0, + block_height: 0, + added_mns: vec![], + removed_mns: vec![], + updated_mns: vec![( + evonode.pro_tx_hash, + diff_entry(serde_json::json!({ "platformNodeID": hex::encode(node_id) })), + )], + }; + let mut at_committed = evonode.clone(); + at_committed.state.platform_node_id = Some([0x86; 20]); + let (platform, committed, _calls) = platform_after_history( + 14, + vec![evonode.clone()], + vec![node_id_change([0x66; 20]), node_id_change([0x86; 20])], + vec![at_committed], + ); + let transaction = platform.drive.grove.start_transaction(); + assert_eq!( + transfer_keys(&platform, &transaction, 0x66, 0x66), + [(vec![0x66; 20], true)], + "PV14 disabled the payout key when the node id left the payout's bytes" + ); + + run_upgrade_events(&platform, &committed, 14, &transaction); + + assert_eq!( + transfer_keys(&platform, &transaction, 0x66, 0x66), + [(vec![0x66; 20], false)] + ); +} + +/// One operator running several masternodes usually gives them one payout address, which +/// protocol version 14 never took from Core's diffs. The first block of protocol version 15 then +/// adds that address's key to each of their operator identities in one batch, which creates the +/// address's hash tree once, also where Drive checks batches for consistency. +#[test] +fn should_reconcile_masternodes_that_share_a_new_payout_in_one_batch() { + for batching_consistency_verification in [false, true] { + let listed = vec![masternode(0x67, 0x67, None), masternode(0x68, 0x68, None)]; + let shared_payout_diff = MasternodeListDiff { + base_height: 0, + block_height: 0, + added_mns: vec![], + removed_mns: vec![], + updated_mns: listed + .iter() + .map(|masternode| { + ( + masternode.pro_tx_hash, + diff_entry(serde_json::json!({ + "operatorPayoutAddress": "yXkZa7p2ne58x2XQYdNthAxfXy9cHiWxF5", + })), + ) + }) + .collect(), + }; + let shared_payout = shared_payout_diff.updated_mns[0] + .1 + .operator_payout_address + .expect("expected the diff to set the payout"); + let at_committed = listed + .iter() + .cloned() + .map(|mut masternode| { + masternode.state.operator_payout_address = shared_payout; + masternode + }) + .collect(); + let mut config = PlatformConfig::default(); + config.drive.batching_consistency_verification = batching_consistency_verification; + let (platform, committed, _calls) = platform_with_config_after_history( + config, + 14, + listed, + vec![shared_payout_diff], + at_committed, + ); + let transaction = platform.drive.grove.start_transaction(); + + run_upgrade_events(&platform, &committed, 14, &transaction); + + let shared_payout = shared_payout.expect("expected a shared payout").to_vec(); + for id in [0x67, 0x68] { + assert_eq!( + transfer_keys(&platform, &transaction, id, id), + [(shared_payout.clone(), false)], + "masternode {id:#x}, consistency checks {batching_consistency_verification}" + ); + } + } +} diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/token_history_storage_flags_tests.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/token_history_storage_flags_tests.rs index 64bac8f699e..ea191bc604d 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/token_history_storage_flags_tests.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/token_history_storage_flags_tests.rs @@ -1,3 +1,4 @@ +use crate::platform_types::platform_state::PlatformState; use crate::rpc::core::MockCoreRPCLike; use crate::test::helpers::setup::{TempPlatform, TestPlatformBuilder}; use dpp::block::block_info::BlockInfo; @@ -34,12 +35,14 @@ fn historical_platform() -> TempPlatform { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(8) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let old = PlatformVersion::get(13).expect("PV13"); let tx = platform.drive.grove.start_transaction(); platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(3), &tx, 8, @@ -134,6 +137,7 @@ fn should_upgrade_historically_owned_token_history_without_version_item_cost_mis let tx = platform.drive.grove.start_transaction(); let result = platform.perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, @@ -399,6 +403,7 @@ fn should_preserve_token_history_flag_owners_and_allocation_epochs() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(epoch), &tx, 13, @@ -488,6 +493,7 @@ fn should_keep_unflagged_and_absent_token_history_unflagged() { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(13) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); if absent { platform @@ -510,6 +516,7 @@ fn should_keep_unflagged_and_absent_token_history_unflagged() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, @@ -546,6 +553,7 @@ fn should_drop_retry_commit_and_reopen_historical_token_history_upgrade() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, @@ -609,11 +617,13 @@ fn should_skip_historical_versions_and_preserve_current_token_history_noop() { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(previous) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let tx = platform.drive.grove.start_transaction(); platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, previous, @@ -642,6 +652,7 @@ fn should_skip_historical_versions_and_preserve_current_token_history_noop() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(8), &tx, pv.protocol_version, @@ -686,6 +697,7 @@ fn should_read_token_history_flag_presence_from_the_candidate_transaction() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, @@ -705,6 +717,7 @@ fn should_preserve_already_current_flagged_token_history_payload() { let platform = TestPlatformBuilder::new() .with_latest_protocol_version() .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let Element::Item(bytes, _) = contract_element(&platform, None) else { panic!("contract Item") @@ -747,6 +760,7 @@ fn should_preserve_already_current_flagged_token_history_payload() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, @@ -771,6 +785,7 @@ fn should_leave_historical_pv13_dispatch_token_history_unchanged() { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block(7), &tx, 13, diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs index 0d0589c0e4d..adc1279ac94 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0/mod.rs @@ -3,6 +3,7 @@ use crate::error::Error; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::PlatformState; use crate::platform_types::platform_state::PlatformStateV0Methods; +use crate::rpc::core::CoreRPCLike; use dpp::block::block_info::BlockInfo; use dpp::data_contracts::SystemDataContract; use dpp::fee::Credits; @@ -46,7 +47,10 @@ use std::borrow::Cow; use std::collections::HashSet; use std::ops::RangeFull; -impl Platform { +impl Platform +where + C: CoreRPCLike, +{ /// Executes protocol-specific events on the first block after a protocol version change. /// /// This function is triggered when there is a protocol version upgrade detected in the network. @@ -69,6 +73,7 @@ impl Platform { pub(super) fn perform_events_on_first_block_of_protocol_change_v0( &self, platform_state: &PlatformState, + block_platform_state: &mut PlatformState, block_info: &BlockInfo, transaction: &Transaction, previous_protocol_version: ProtocolVersion, @@ -122,7 +127,12 @@ impl Platform { } if previous_protocol_version < 15 && platform_version.protocol_version >= 15 { - self.transition_to_version_15(block_info, transaction, platform_version)?; + self.transition_to_version_15( + block_platform_state, + block_info, + transaction, + platform_version, + )?; } Ok(()) @@ -917,14 +927,34 @@ impl Platform { /// trigger, re-store the DashPay contract whose v3 schema states the checks of /// both its data triggers, create the vote decisions trees that yes/no polls /// live in, insert the decision proposals contract whose proposals open those - /// polls, and add the token marketplace: its escrow trees and the token market - /// contract. + /// polls, add the token marketplace: its escrow trees and the token market + /// contract, and reconcile the operator payouts of Platform's masternode list + /// with Dash Core's. fn transition_to_version_15( &self, + block_platform_state: &mut PlatformState, block_info: &BlockInfo, transaction: &Transaction, platform_version: &PlatformVersion, ) -> Result<(), Error> { + // Operator payouts (v15 note 28): from this version they follow Core's diffs, and this + // fixes once what earlier versions left stale. Reconciled against Core's list at the + // last committed chain locked core height, the height the masternode list is at here; + // this block's own diff to its chain locked height is applied afterwards, under this + // version's rules. That height is chain locked and already processed, so every node's + // Core holds it and reports the same list. This puts Core on the upgrade block's path + // whatever the block's own height: a node whose Core cannot answer cannot execute the + // upgrade block. It goes first, so a round whose Core cannot answer stops before the + // rest of this transition's work. + let core_height = block_platform_state.last_committed_core_height(); + self.reconcile_operator_payouts_with_core( + block_platform_state, + core_height, + block_info, + transaction, + platform_version, + )?; + // DPNS contract v3 (v15 note 1): its preorders are unsalted, one per owner and name, // which no contract update can turn a unique index into, so the `preorder` type is // rebuilt and every preorder made before this block is deleted, unrefunded. @@ -1194,6 +1224,7 @@ mod tests { // When previous == current, no transition_to_version_* should be triggered let result = platform.perform_events_on_first_block_of_protocol_change_v0( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, platform_version.protocol_version, @@ -1894,6 +1925,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_version_14 = PlatformVersion::get(14).expect("expected platform version 14"); @@ -1933,7 +1965,12 @@ mod tests { }; platform - .transition_to_version_15(&block_info, &transaction, platform_version) + .transition_to_version_15( + &mut PlatformState::clone(&platform.state.load()), + &block_info, + &transaction, + platform_version, + ) .expect("expected the transition to succeed"); let stored = platform @@ -2342,6 +2379,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, 13, @@ -2455,6 +2493,7 @@ mod tests { let mut platform = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); @@ -2598,6 +2637,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, 14, @@ -2997,10 +3037,12 @@ mod tests { let born_at_15 = TestPlatformBuilder::new() .with_initial_protocol_version(15) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let upgraded = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let transaction = upgraded.drive.grove.start_transaction(); @@ -3026,7 +3068,12 @@ mod tests { } upgraded - .transition_to_version_15(&BlockInfo::default(), &transaction, platform_version) + .transition_to_version_15( + &mut PlatformState::clone(&upgraded.state.load()), + &BlockInfo::default(), + &transaction, + platform_version, + ) .expect("expected version 15 transition to succeed"); for key in [ACTIVE_POLLS_TREE_KEY, IDENTITY_VOTES_TREE_KEY] { @@ -3215,10 +3262,12 @@ mod tests { let born_at_15 = TestPlatformBuilder::new() .with_initial_protocol_version(15) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let upgraded = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let transaction = upgraded.drive.grove.start_transaction(); @@ -3229,7 +3278,12 @@ mod tests { epoch: Epoch::new(1).expect("expected epoch"), }; upgraded - .transition_to_version_15(&block_info, &transaction, platform_version) + .transition_to_version_15( + &mut PlatformState::clone(&upgraded.state.load()), + &block_info, + &transaction, + platform_version, + ) .expect("expected version 15 transition to succeed"); let element = |platform: &Platform, @@ -3570,6 +3624,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(10) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let transaction = platform.drive.grove.start_transaction(); @@ -3586,6 +3641,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change_v0( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &transaction, 10, @@ -4531,11 +4587,13 @@ mod tests { let platform_a = TestPlatformBuilder::new() .with_initial_protocol_version(15) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_b = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); // A genuine v14 genesis has neither escrow tree yet. @@ -4565,7 +4623,12 @@ mod tests { let txn_b = platform_b.drive.grove.start_transaction(); platform_b - .transition_to_version_15(&BlockInfo::default(), &txn_b, platform_version_15) + .transition_to_version_15( + &mut PlatformState::clone(&platform_b.state.load()), + &BlockInfo::default(), + &txn_b, + platform_version_15, + ) .expect("upgrade: transition_to_version_15 should succeed"); let roots = [ @@ -4665,6 +4728,7 @@ mod tests { let platform = TestPlatformBuilder::new() .with_initial_protocol_version(14) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_version_14 = PlatformVersion::get(14).expect("expected platform version 14"); @@ -4690,7 +4754,12 @@ mod tests { ); platform - .transition_to_version_15(&BlockInfo::default(), &transaction, platform_version) + .transition_to_version_15( + &mut PlatformState::clone(&platform.state.load()), + &BlockInfo::default(), + &transaction, + platform_version, + ) .expect("expected the transition to succeed"); let stored = platform diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v1/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v1/mod.rs index e6dd702b16e..26f16bc09a7 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v1/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v1/mod.rs @@ -1,13 +1,17 @@ use crate::error::Error; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::PlatformState; +use crate::rpc::core::CoreRPCLike; use dpp::block::block_info::BlockInfo; use dpp::data_contracts::SystemDataContract; use dpp::version::PlatformVersion; use dpp::version::ProtocolVersion; use drive::grovedb::Transaction; -impl Platform { +impl Platform +where + C: CoreRPCLike, +{ /// Runs the protocol change events and then refreshes the cached definitions of the system /// contracts those events may have rewritten. /// @@ -32,6 +36,7 @@ impl Platform { pub(super) fn perform_events_on_first_block_of_protocol_change_v1( &self, platform_state: &PlatformState, + block_platform_state: &mut PlatformState, block_info: &BlockInfo, transaction: &Transaction, previous_protocol_version: ProtocolVersion, @@ -39,6 +44,7 @@ impl Platform { ) -> Result<(), Error> { self.perform_events_on_first_block_of_protocol_change_v0( platform_state, + block_platform_state, block_info, transaction, previous_protocol_version, diff --git a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v2/mod.rs b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v2/mod.rs index c19d89cc8d5..ce991c55e05 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v2/mod.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/protocol_upgrade/perform_events_on_first_block_of_protocol_change/v2/mod.rs @@ -1,12 +1,16 @@ use crate::error::Error; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::PlatformState; +use crate::rpc::core::CoreRPCLike; use dpp::block::block_info::BlockInfo; use dpp::version::PlatformVersion; use dpp::version::ProtocolVersion; use drive::grovedb::Transaction; -impl Platform { +impl Platform +where + C: CoreRPCLike, +{ /// Empties the contract cache, then runs the protocol change events of v1. /// /// A cached contract can carry the fee of the read that cached it, calculated under the fee @@ -22,6 +26,7 @@ impl Platform { pub(super) fn perform_events_on_first_block_of_protocol_change_v2( &self, platform_state: &PlatformState, + block_platform_state: &mut PlatformState, block_info: &BlockInfo, transaction: &Transaction, previous_protocol_version: ProtocolVersion, @@ -31,6 +36,7 @@ impl Platform { self.perform_events_on_first_block_of_protocol_change_v1( platform_state, + block_platform_state, block_info, transaction, previous_protocol_version, diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/creation.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/creation.rs index 6abda31b596..872998d6614 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/creation.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/creation.rs @@ -1,6 +1,7 @@ use super::*; mod creation_tests { + use crate::platform_types::platform_state::PlatformState; use super::*; use dapi_grpc::platform::v0::{get_contested_resource_vote_state_request, get_contested_resource_vote_state_response, GetContestedResourceVoteStateRequest, GetContestedResourceVoteStateResponse}; use dapi_grpc::platform::v0::get_contested_resource_vote_state_request::get_contested_resource_vote_state_request_v0::ResultType; @@ -3964,6 +3965,7 @@ mod creation_tests { let mut platform = TestPlatformBuilder::new() .with_initial_protocol_version(13) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); @@ -3986,6 +3988,7 @@ mod creation_tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &BlockInfo::default_with_time( platform_state .last_committed_block_time_ms() diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns.rs index 16a61100e25..2df11542360 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns.rs @@ -1031,6 +1031,7 @@ mod dpns_tests { mod dpns_username_transfer_tests { use super::*; + use crate::platform_types::platform_state::PlatformState; use crate::test::helpers::dpns::set_dpns_preorder_hashes; use dpp::consensus::basic::BasicError; use dpp::consensus::state::data_trigger::DataTriggerError; @@ -1584,6 +1585,7 @@ mod dpns_username_transfer_tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &rejected_transaction, 12, @@ -1628,6 +1630,7 @@ mod dpns_username_transfer_tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block_info, &retry_transaction, 12, @@ -1683,6 +1686,7 @@ mod dpns_username_transfer_tests { clean_control .perform_events_on_first_block_of_protocol_change( &clean_control_state, + &mut PlatformState::clone(&clean_control_state), &block_info, &clean_control_transaction, 12, @@ -1934,6 +1938,7 @@ mod dpns_username_transfer_tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), block_info, &transaction, 12, @@ -2002,6 +2007,7 @@ mod dpns_username_transfer_tests { platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &v13_activation_block_info(), &transaction, 12, diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns_v3.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns_v3.rs index 4ce5ba13fa3..6c7e8a8c976 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns_v3.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/dpns_v3.rs @@ -18,6 +18,7 @@ mod dpns_v3_tests { use crate::execution::validation::state_transition::tests::{ create_dpns_name_contest_give_key_info, perform_votes_multi, }; + use crate::platform_types::platform_state::PlatformState; use crate::rpc::core::MockCoreRPCLike; use crate::test::helpers::dpns::dpns_preorder_properties; use crate::test::helpers::fast_forward_to_block::fast_forward_to_block; @@ -98,6 +99,7 @@ mod dpns_v3_tests { let mut platform = TestPlatformBuilder::new() .with_initial_protocol_version(protocol_version) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let mut writer = |seed| { @@ -413,6 +415,7 @@ mod dpns_v3_tests { self.platform .perform_events_on_first_block_of_protocol_change( &platform_state, + &mut PlatformState::clone(&platform_state), &block(height), &transaction, previous, @@ -1249,6 +1252,7 @@ mod dpns_v3_tests { let born_at_15 = TestPlatformBuilder::new() .with_initial_protocol_version(15) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); assert_eq!( preorder_type_layout(&fixture.platform, fixture.platform_version), @@ -1310,6 +1314,7 @@ mod dpns_v3_tests { let born_at_15 = TestPlatformBuilder::new() .with_initial_protocol_version(15) .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); assert_eq!( preorder_type_layout(&fixture.platform, fixture.platform_version), @@ -1409,6 +1414,7 @@ mod dpns_v3_tests { let mut platform = TestPlatformBuilder::new() .with_latest_protocol_version() .build_with_mock_rpc() + .with_no_core_masternodes() .set_genesis_state(); let platform_state = platform.state.load(); diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/distribution/once_per_identity.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/distribution/once_per_identity.rs index 394d27f3e60..4a456ca6103 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/distribution/once_per_identity.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/distribution/once_per_identity.rs @@ -14,6 +14,7 @@ use rand::prelude::StdRng; /// the total paid out is bounded only by the token's max supply. mod once_per_identity_distribution { use super::*; + use crate::platform_types::platform_state::PlatformState; use crate::platform_types::platform_state::PlatformStateV0Methods; use crate::platform_types::state_transitions_processing_result::StateTransitionExecutionResult; use crate::rpc::core::MockCoreRPCLike; @@ -756,6 +757,7 @@ mod once_per_identity_distribution { platform .perform_events_on_first_block_of_protocol_change( &platform.state.load(), + &mut PlatformState::clone(&platform.state.load()), &block_info(50, 40), &transaction, 13, diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/shield/tests.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/shield/tests.rs index 350822bf4c1..9e1bb3b5629 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/shield/tests.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/shield/tests.rs @@ -1418,6 +1418,7 @@ mod tests { insert_anchor_into_state, serialize_authorized_bundle_u64, }; use crate::platform_types::platform::PlatformRef; + use crate::platform_types::platform_state::PlatformState; use crate::platform_types::platform_state::PlatformStateV0Methods; use crate::rpc::core::MockCoreRPCLike; use crate::test::helpers::setup::TempPlatform; @@ -1585,7 +1586,7 @@ mod tests { ) { let old = PlatformVersion::get(13).expect("PV13"); let latest = PlatformVersion::latest(); - let mut platform = setup_platform_at_protocol_version(13); + let mut platform = setup_platform_at_protocol_version(13).with_no_core_masternodes(); let (signer, address) = funded_address(&mut platform); let (historical, mut notes) = historical_bundle_with_notes(1_000_000_000, &[], 13014); let shield = signed_shield(&signer, address, 1, &historical).await; @@ -1623,6 +1624,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &state, + &mut PlatformState::clone(&state), &BlockInfo::default(), &tx, 13, @@ -1668,7 +1670,7 @@ mod tests { async fn should_refuse_historical_credit_rho_reuse_on_first_pv14_candidate() { let old_version = PlatformVersion::get(13).expect("protocol 13"); let new_version = PlatformVersion::latest(); - let mut platform = setup_platform_at_protocol_version(13); + let mut platform = setup_platform_at_protocol_version(13).with_no_core_masternodes(); let (signer, address) = funded_address(&mut platform); let historical = historical_bundle(&[], 14013); let first = signed_shield(&signer, address, 1, &historical).await; @@ -1737,6 +1739,7 @@ mod tests { platform .perform_events_on_first_block_of_protocol_change( &candidate_state, + &mut PlatformState::clone(&candidate_state), &BlockInfo::default(), &candidate, 13, diff --git a/packages/rs-drive-abci/src/main.rs b/packages/rs-drive-abci/src/main.rs index e21a81244eb..1a7f38346d4 100644 --- a/packages/rs-drive-abci/src/main.rs +++ b/packages/rs-drive-abci/src/main.rs @@ -353,10 +353,12 @@ fn dump_config(config: &PlatformConfig) -> Result<(), String> { #[cfg(feature = "shielded_test_data")] mod snapshot_bake_main { use dpp::dashcore::ephemerealdata::chain_lock::ChainLock; - use dpp::dashcore::{Block, BlockHash, Header, InstantLock, QuorumHash, Transaction, Txid}; + use dpp::dashcore::{ + Block, BlockHash, Header, InstantLock, ProTxHash, QuorumHash, Transaction, Txid, + }; use dpp::dashcore_rpc::dashcore_rpc_json::{ - AssetUnlockStatusResult, ExtendedQuorumListResult, GetChainTipsResult, MasternodeListDiff, - MnSyncStatus, QuorumInfoResult, QuorumType, SoftforkInfo, + AssetUnlockStatusResult, DMNState, ExtendedQuorumListResult, GetChainTipsResult, + MasternodeListDiff, MnSyncStatus, QuorumInfoResult, QuorumType, SoftforkInfo, }; use dpp::dashcore_rpc::json::GetRawTransactionResult; use dpp::dashcore_rpc::Error; @@ -457,6 +459,9 @@ mod snapshot_bake_main { fn get_coinbase_payload_version(&self, _: u32) -> Result, Error> { unreachable!() } + fn get_masternode_state(&self, _: &ProTxHash, _: u32) -> Result { + unreachable!() + } } /// Produce a shielded-pool snapshot at `out_path` from a fresh temporary diff --git a/packages/rs-drive-abci/src/rpc/core.rs b/packages/rs-drive-abci/src/rpc/core.rs index 2208254390d..c63ef5a49b4 100644 --- a/packages/rs-drive-abci/src/rpc/core.rs +++ b/packages/rs-drive-abci/src/rpc/core.rs @@ -3,11 +3,12 @@ use dpp::dashcore::consensus::encode::deserialize_partial; use dpp::dashcore::ephemerealdata::chain_lock::ChainLock; use dpp::dashcore::transaction::special_transaction::coinbase::CoinbasePayload; use dpp::dashcore::transaction::special_transaction::TransactionPayload; -use dpp::dashcore::{Block, BlockHash, QuorumHash, Transaction, Txid}; +use dpp::dashcore::{Block, BlockHash, ProTxHash, QuorumHash, Transaction, Txid}; use dpp::dashcore::{Header, InstantLock}; use dpp::dashcore_rpc::dashcore_rpc_json::{ - AssetUnlockStatusResult, ExtendedQuorumDetails, ExtendedQuorumListResult, GetChainTipsResult, - MasternodeListDiff, MnSyncStatus, QuorumInfoResult, QuorumType, SoftforkInfo, + AssetUnlockStatusResult, DMNState, ExtendedQuorumDetails, ExtendedQuorumListResult, + GetChainTipsResult, MasternodeListDiff, MnSyncStatus, QuorumInfoResult, QuorumType, + SoftforkInfo, }; use dpp::dashcore_rpc::json::GetRawTransactionResult; use dpp::dashcore_rpc::jsonrpc::error::{Error as JsonRpcError, RpcError}; @@ -179,8 +180,14 @@ pub trait CoreRPCLike { block: u32, ) -> Result; - // /// Get the detailed information about a deterministic masternode - // fn get_protx_info(&self, pro_tx_hash: &ProTxHash) -> Result; + /// Get the state of the masternode `pro_tx_hash` in Core's masternode list at the block at + /// `height` (`protx info` at that block's hash). Only ask for a chain locked height: the + /// answer is then the same on every node. + fn get_masternode_state( + &self, + pro_tx_hash: &ProTxHash, + height: CoreHeight, + ) -> Result; /// Verify Instant Lock signature /// If `max_height` is provided the chain lock will be verified @@ -550,18 +557,67 @@ impl CoreRPCLike for DefaultCoreRPC { }; coinbase_payload_version_from_coinbase(&coinbase).map_err(Error::UnexpectedStructure) } + + fn get_masternode_state( + &self, + pro_tx_hash: &ProTxHash, + height: CoreHeight, + ) -> Result { + let block_hash = self.get_block_hash(height)?; + let args = [ + Value::String("info".to_string()), + // Core's form of the hash. `ProTxHash::to_hex` reverses the bytes, and Core then + // finds no such masternode. + Value::String(pro_tx_hash.to_string()), + Value::String(block_hash.to_string()), + ]; + let answer = retry!(self.inner.call::("protx", &args))?; + masternode_state_from_protx_info(pro_tx_hash, answer) + } +} + +/// The `state` of Core's `protx info` answer for `pro_tx_hash`, read on its own. +/// +/// The rest of the answer is not read: it carries fields the RPC client's typed answer cannot +/// read for every masternode (`operatorReward` is a decimal such as `13.37`, read there as an +/// integer) or that differ between nodes (`metaInfo`), and a failed read would fail the block. +/// The state is the one `protx listdiff` lists, read the same way. +fn masternode_state_from_protx_info( + pro_tx_hash: &ProTxHash, + mut answer: Value, +) -> Result { + let answered_for = answer.get("proTxHash").and_then(Value::as_str); + if answered_for != Some(pro_tx_hash.to_string().as_str()) { + return Err(Error::UnexpectedStructure(format!( + "protx info for {pro_tx_hash} answered for {answered_for:?}" + ))); + } + let state = answer.get_mut("state").map(Value::take).ok_or_else(|| { + Error::UnexpectedStructure(format!("protx info for {pro_tx_hash} has no state")) + })?; + serde_json::from_value(state).map_err(|error| { + Error::UnexpectedStructure(format!( + "protx info for {pro_tx_hash} has a state that cannot be read: {error}" + )) + }) } #[cfg(test)] mod tests { - use super::{coinbase_payload_version_from_coinbase, credit_pool_balance_from_coinbase}; + use super::{ + coinbase_payload_version_from_coinbase, credit_pool_balance_from_coinbase, + masternode_state_from_protx_info, + }; use dpp::dashcore::bls_sig_utils::BLSSignature; use dpp::dashcore::consensus::serialize; use dpp::dashcore::hash_types::{MerkleRootMasternodeList, MerkleRootQuorums}; use dpp::dashcore::hashes::Hash; use dpp::dashcore::transaction::special_transaction::coinbase::CoinbasePayload; use dpp::dashcore::transaction::special_transaction::TransactionPayload; + use dpp::dashcore::ProTxHash; use dpp::dashcore::{OutPoint, ScriptBuf, Transaction, TxIn, TxOut}; + use serde_json::json; + use std::str::FromStr; const BALANCE_DUFFS: u64 = 3_700_000_000_000; @@ -712,13 +768,17 @@ mod tests { /// answers recorded from Dash Core v24.0.0-rc.3 and v23.1.8. mod fork_info { use crate::rpc::core::{CoreRPCLike, DefaultCoreRPC}; - use dpp::dashcore_rpc::dashcore_rpc_json::{SoftforkInfo, SoftforkType}; + use dpp::dashcore::ProTxHash; + use dpp::dashcore_rpc::dashcore_rpc_json::{ + MasternodeListDiff, SoftforkInfo, SoftforkType, + }; use dpp::dashcore_rpc::jsonrpc::error::{Error as JsonRpcError, RpcError}; use dpp::dashcore_rpc::jsonrpc::simple_http::Error as HttpError; use dpp::dashcore_rpc::Error; use serde_json::{json, Value}; use std::io::{BufRead, BufReader, Read, Write}; use std::net::{TcpListener, TcpStream}; + use std::str::FromStr; use std::sync::{Arc, Mutex}; use std::thread; @@ -820,6 +880,7 @@ mod tests { struct MockCore { url: String, asked: Arc>>, + params: Arc>>, } impl MockCore { @@ -832,17 +893,20 @@ mod tests { listener.local_addr().expect("mock core address") ); let asked = Arc::new(Mutex::new(Vec::new())); + let params = Arc::new(Mutex::new(Vec::new())); let recorder = Arc::clone(&asked); + let params_recorder = Arc::clone(¶ms); let answers = Arc::new(Mutex::new(answers)); thread::spawn(move || { for stream in listener.incoming() { let Ok(stream) = stream else { return }; let answers = Arc::clone(&answers); let recorder = Arc::clone(&recorder); - thread::spawn(move || serve(stream, &answers, &recorder)); + let params_recorder = Arc::clone(¶ms_recorder); + thread::spawn(move || serve(stream, &answers, &recorder, ¶ms_recorder)); } }); - MockCore { url, asked } + MockCore { url, asked, params } } fn rpc(&self) -> DefaultCoreRPC { @@ -854,6 +918,17 @@ mod tests { fn asked(&self) -> Vec { self.asked.lock().expect("asked lock").clone() } + + /// The params of every request for `method` so far, in order. + fn params_of(&self, method: &str) -> Vec { + self.params + .lock() + .expect("params lock") + .iter() + .filter(|(asked, _)| asked == method) + .map(|(_, params)| params.clone()) + .collect() + } } /// The answer to `method`, moving on to the method's next entry if it has one. @@ -879,6 +954,7 @@ mod tests { stream: TcpStream, answers: &Mutex>, asked: &Mutex>, + params: &Mutex>, ) { let mut writer = stream.try_clone().expect("clone mock core stream"); let mut reader = BufReader::new(stream); @@ -909,6 +985,10 @@ mod tests { .expect("request method") .to_string(); asked.lock().expect("asked lock").push(method.clone()); + params + .lock() + .expect("params lock") + .push((method.clone(), request["params"].clone())); let json = |text: &str| serde_json::from_str::(text).expect("answer is JSON"); @@ -943,6 +1023,76 @@ mod tests { } } + /// A block hash `getblockhash` answers, as Core prints it. + const BLOCK_HASH: &str = + "\"74b4fb2caeeebbc7cd5dcc50b58532d983f0e5c87f3ac729c6d47f0ace76b97d\""; + + /// MN_L2 of the legacy operator key recording, as Core prints its hash. + const MN_L2: &str = "02cc32acbd70020b192abdddfd8ab6d682b0e75c6cbe225f60fe6cf9da08859b"; + + /// A `protx info` answer for MN_L2 as Dash Core prints one, with an operator reward that + /// is not a whole percentage and the node's own meta info. + const PROTX_INFO: &str = r#"{ + "type": "Regular", + "proTxHash": "02cc32acbd70020b192abdddfd8ab6d682b0e75c6cbe225f60fe6cf9da08859b", + "collateralHash": "2d3a9f3c4cfa3dfba1b2b2b6f6e0e4cbbd2f6a6a2f7f8b5f4c3a8b0c1d2e3f40", + "collateralIndex": 0, + "collateralAddress": "yMY3XstUm4p29eX3GLuRggyy9o8tQhp4HP", + "operatorReward": 13.37, + "state": {"version": 1, "service": "127.0.0.1:30002", "addresses": {"core_p2p": ["127.0.0.1:30002"]}, "registeredHeight": 302, "lastPaidHeight": 398, "consecutivePayments": 0, "PoSePenalty": 0, "PoSeRevivedHeight": -1, "PoSeBanHeight": -1, "revocationReason": 0, "ownerAddress": "yhKZ8Zy1243u88wFujwkScuW9Lkomo2NHZ", "votingAddress": "yhUTNhZjweRvHEJ43JUefpTuHavmDh3LjP", "payoutAddress": "yXCxDa7nTBA31La9SVxh2Dr5k9BrF6JBcJ", "pubKeyOperator": "910633a672392e26860cbff612b28d84f71beb34c2092d015b0d522905d628d38d746c1088c654632c65c6a7f008095d", "operatorPayoutAddress": "yMY3XstUm4p29eX3GLuRggyy9o8tQhp4HP"}, + "confirmations": 48, + "metaInfo": { + "lastDSQ": 0, + "mixingTxCount": 0, + "outboundAttemptCount": 0, + "lastOutboundAttempt": 0, + "lastOutboundAttemptElapsed": 1762400000, + "lastOutboundSuccess": 0, + "lastOutboundSuccessElapsed": 1762400000 + } + }"#; + + /// The state of a masternode at a Core height is `protx info` for the masternode's hash + /// in Core's form at the hash of the block at that height, and its state is read whatever + /// the rest of the answer holds. + #[test] + fn should_ask_protx_info_for_the_masternode_at_the_block_of_the_height() { + let core = MockCore::start(vec![ + ("getblockhash", Answer::Result(BLOCK_HASH)), + ("protx", Answer::Result(PROTX_INFO)), + ]); + // As Platform has the hash: decoded from Core's masternode list. + let listed: MasternodeListDiff = serde_json::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/tests/supporting_files/core_v24_protx_list_diffs/legacy_operator_key/1-399.json" + ))) + .expect("expected the recorded masternode list"); + let pro_tx_hash = listed + .added_mns + .iter() + .map(|masternode| masternode.pro_tx_hash) + .find(|pro_tx_hash| { + *pro_tx_hash == ProTxHash::from_str(MN_L2).expect("expected a pro tx hash") + }) + .expect("expected MN_L2 in the recording"); + + let state = core + .rpc() + .get_masternode_state(&pro_tx_hash, 401) + .expect("expected the masternode's state"); + + assert!(state.operator_payout_address.is_some()); + assert_eq!(core.params_of("getblockhash"), [json!([401])]); + assert_eq!( + core.params_of("protx"), + [json!([ + "info", + MN_L2, + "74b4fb2caeeebbc7cd5dcc50b58532d983f0e5c87f3ac729c6d47f0ace76b97d" + ])] + ); + } + fn active_mn_rr() -> SoftforkInfo { SoftforkInfo { softfork_type: SoftforkType::Buried, @@ -1086,4 +1236,71 @@ mod tests { assert_eq!(core.asked(), ["getdeploymentinfo"]); } } + + /// A `protx info` answer as Dash Core prints it, for MN_L2 of the legacy operator key + /// recording, with an operator reward that is not a whole percentage. + fn protx_info_answer(pro_tx_hash: &str) -> serde_json::Value { + let recording: serde_json::Value = serde_json::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/tests/supporting_files/core_v24_protx_list_diffs/legacy_operator_key/1-399.json" + ))) + .expect("expected the recording"); + let listed = recording["addedMNs"] + .as_array() + .expect("expected the listed masternodes") + .iter() + .find(|masternode| masternode["proTxHash"] == pro_tx_hash) + .expect("expected the masternode in the recording") + .clone(); + json!({ + "type": listed["type"], + "proTxHash": pro_tx_hash, + "collateralHash": listed["collateralHash"], + "collateralIndex": listed["collateralIndex"], + "collateralAddress": listed["collateralAddress"], + "operatorReward": 13.37, + "state": listed["state"], + "confirmations": 12, + "metaInfo": { + "lastDSQ": 0, + "mixingTxCount": 0, + "outboundAttemptCount": 0, + "lastOutboundAttempt": 0, + "lastOutboundAttemptElapsed": 1762400000, + "lastOutboundSuccess": 0, + "lastOutboundSuccessElapsed": 1762400000, + }, + }) + } + + const MN_L2: &str = "02cc32acbd70020b192abdddfd8ab6d682b0e75c6cbe225f60fe6cf9da08859b"; + + /// The state is read whatever the rest of the answer holds: an operator reward of 13.37 % + /// and the node's own meta info, which the typed answer of the RPC client cannot read for + /// every masternode, fail nothing. + #[test] + fn should_read_the_state_of_a_protx_info_answer() { + let pro_tx_hash = ProTxHash::from_str(MN_L2).expect("expected a pro tx hash"); + let state = masternode_state_from_protx_info(&pro_tx_hash, protx_info_answer(MN_L2)) + .expect("expected to read the state"); + assert!(state.operator_payout_address.is_some()); + assert_eq!(state.pub_key_operator.len(), 48); + } + + /// An answer for another masternode, or without a state, is not read as this one's. + #[test] + fn should_refuse_a_protx_info_answer_for_another_masternode_or_without_a_state() { + let pro_tx_hash = ProTxHash::from_str(MN_L2).expect("expected a pro tx hash"); + let other = "f18c25367cd4f2ba6e8cdf86d0ffcf2694b6206b75597fe1ab5b80b966dbdf4b"; + let mut for_other = protx_info_answer(MN_L2); + for_other["proTxHash"] = json!(other); + let mut without_state = protx_info_answer(MN_L2); + without_state + .as_object_mut() + .expect("expected an object") + .remove("state"); + for answer in [for_other, without_state] { + assert!(masternode_state_from_protx_info(&pro_tx_hash, answer).is_err()); + } + } } diff --git a/packages/rs-drive-abci/src/test/helpers/setup.rs b/packages/rs-drive-abci/src/test/helpers/setup.rs index 35158b37d32..91dbfe3d04e 100644 --- a/packages/rs-drive-abci/src/test/helpers/setup.rs +++ b/packages/rs-drive-abci/src/test/helpers/setup.rs @@ -11,6 +11,7 @@ use crate::platform_types::platform_state::PlatformStateV0Methods; use crate::rpc::core::MockCoreRPCLike; use crate::{config::PlatformConfig, rpc::core::DefaultCoreRPC}; use dpp::block::block_info::BlockInfo; +use dpp::dashcore_rpc::dashcore_rpc_json::MasternodeListDiff; use dpp::document::transfer::Transferable; use dpp::nft::TradeMode; use dpp::prelude::{CoreBlockHeight, DataContract, TimestampMillis}; @@ -200,6 +201,28 @@ impl TempPlatform { (self, card_game_contract) } + /// Has the mock Core list no masternodes at any height, which is the list of a platform that + /// never processed one. The first block of protocol version 15 reads Core's masternode list + /// to reconcile operator payouts. It answers every full list call, and the mock takes the + /// first expectation that matches, so a test that answers them itself must not use it. A + /// diff between two heights still finds no answer. + pub fn with_no_core_masternodes(mut self) -> Self { + self.platform + .core_rpc + .expect_get_protx_diff_with_masternodes() + .withf(|base_block, _| base_block.is_none()) + .returning(|base_block, block| { + Ok(MasternodeListDiff { + base_height: base_block.unwrap_or_default(), + block_height: block, + added_mns: vec![], + removed_mns: vec![], + updated_mns: vec![], + }) + }); + self + } + /// Sets Platform to genesis state. pub fn set_genesis_state(self) -> Self { let platform_state = self.platform.state.load(); diff --git a/packages/rs-drive-abci/tests/strategy_tests/execution.rs b/packages/rs-drive-abci/tests/strategy_tests/execution.rs index e59234f9aed..d6c6eeb6335 100644 --- a/packages/rs-drive-abci/tests/strategy_tests/execution.rs +++ b/packages/rs-drive-abci/tests/strategy_tests/execution.rs @@ -24,6 +24,7 @@ use dpp::dashcore::consensus::Encodable; use dpp::dashcore::hashes::{sha256d, HashEngine}; use dpp::dashcore::{ChainLock, QuorumSigningRequestId, VarInt}; use dpp::dashcore_rpc::json::{ExtendedQuorumListResult, SoftforkInfo}; +use dpp::dashcore_rpc::Error as CoreRpcError; use drive::structure::conformance::check_conformance; use drive::structure::drive_structure; use drive_abci::abci::app::FullAbciApplication; @@ -507,18 +508,64 @@ pub(crate) async fn run_chain_for_strategy<'a>( .clone()) }); + // Core's state of one masternode at a height, as the strategy made it. From protocol version + // 15 a block reads it for an operator key change printed without a payout. + let masternodes_by_height = with_extra_masternodes_with_updates.clone(); + let hpmns_by_height = with_extra_hpmns_with_updates.clone(); + let initial_masternodes = initial_all_masternodes.clone(); + platform + .core_rpc + .expect_get_masternode_state() + .returning(move |pro_tx_hash, block| { + let at_height = masternodes_by_height + .get(&block) + .into_iter() + .chain(hpmns_by_height.get(&block)) + .flatten() + .find(|masternode| masternode.pro_tx_hash() == *pro_tx_hash) + .map(|masternode| masternode.get_state_at_height(block).state.clone()); + at_height + .or_else(|| { + initial_masternodes + .iter() + .find(|masternode| masternode.pro_tx_hash() == *pro_tx_hash) + .map(|masternode| masternode.get_state_at_height(block).state.clone()) + }) + .ok_or_else(|| { + CoreRpcError::UnexpectedStructure(format!( + "no masternode {pro_tx_hash} in the strategy at core height {block}" + )) + }) + }); + + let genesis_core_height = config.abci.genesis_core_height; platform .core_rpc .expect_get_protx_diff_with_masternodes() .returning(move |base_block, block| { let diff = if base_block.is_none() { - MasternodeListDiff { - base_height: 0, - block_height: block, - added_mns: initial_all_masternodes + // Core's whole list at `block`: the initial masternodes at the genesis core + // height, where init chain starts, and what the strategy made of them at a later + // height, which the first block of protocol version 15 reads to reconcile + // operator payouts. + let added_mns = match ( + with_extra_masternodes_with_updates.get(&block), + with_extra_hpmns_with_updates.get(&block), + ) { + (Some(masternodes), Some(hpmns)) if block != genesis_core_height => masternodes + .iter() + .chain(hpmns) + .map(|masternode| masternode.get_state_at_height(block).clone()) + .collect(), + _ => initial_all_masternodes .iter() .map(|masternode_list_item| masternode_list_item.masternode.clone()) .collect(), + }; + MasternodeListDiff { + base_height: 0, + block_height: block, + added_mns, removed_mns: vec![], updated_mns: vec![], } diff --git a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/mod.rs index 187abcfaf24..d8f13cde3bc 100644 --- a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/mod.rs +++ b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/mod.rs @@ -63,56 +63,4 @@ impl Drive { })), } } - - /// The operations for adding new non-unique keys to a masternode identity. This function is - /// version controlled by the version of `add_new_keys_to_identity_operations`, whose - /// operations it emits when estimating costs and for every key whose hash is not a unique - /// key hash. - /// - /// A key whose hash is already a unique key hash is added without a non-unique key hash - /// reference, matching how keys are stored when a masternode identity is created. - /// - /// # Arguments - /// - /// * `identity_id` - The ID of the masternode identity to which keys are to be added. - /// * `non_unique_keys_to_add` - The keys to be added, all of non-unique key types. - /// * `epoch` - The current epoch. - /// * `estimated_costs_only_with_layer_info` - The estimated costs with layer information. - /// * `transaction` - The current transaction. - /// * `platform_version` - The platform version. - /// - /// # Returns - /// - /// * `Result, Error>` - The resulting low level drive operations if successful, or an error. - pub fn add_new_keys_to_masternode_identity_operations( - &self, - identity_id: [u8; 32], - non_unique_keys_to_add: Vec, - epoch: &Epoch, - estimated_costs_only_with_layer_info: &mut Option, - transaction: TransactionArg, - platform_version: &PlatformVersion, - ) -> Result, Error> { - match platform_version - .drive - .methods - .identity - .update - .add_new_keys_to_identity - { - 0 => self.add_new_keys_to_masternode_identity_operations_v0( - identity_id, - non_unique_keys_to_add, - epoch, - estimated_costs_only_with_layer_info, - transaction, - platform_version, - ), - version => Err(Error::Drive(DriveError::UnknownVersionMismatch { - method: "add_new_keys_to_masternode_identity_operations".to_string(), - known_versions: vec![0], - received: version, - })), - } - } } diff --git a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/v0/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/v0/mod.rs index 85b03991ba7..8f9c92b559a 100644 --- a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/v0/mod.rs +++ b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_identity/v0/mod.rs @@ -2,7 +2,6 @@ use crate::drive::Drive; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; use crate::util::estimated_layers::EstimatedLayers; -use dpp::identity::identity_public_key::methods::hash::IdentityPublicKeyHashMethodsV0; use dpp::identity::IdentityPublicKey; use dpp::block::epoch::Epoch; @@ -62,51 +61,4 @@ impl Drive { } Ok(drive_operations) } - - /// The operations for adding new non-unique keys to a masternode identity - #[inline(always)] - pub(super) fn add_new_keys_to_masternode_identity_operations_v0( - &self, - identity_id: [u8; 32], - non_unique_keys_to_add: Vec, - epoch: &Epoch, - estimated_costs_only_with_layer_info: &mut Option, - transaction: TransactionArg, - platform_version: &PlatformVersion, - ) -> Result, Error> { - let mut drive_operations: Vec = vec![]; - if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info { - Self::add_estimation_costs_for_keys_for_identity_id( - identity_id, - estimated_costs_only_with_layer_info, - &platform_version.drive, - )?; - } - - for key in non_unique_keys_to_add { - // When estimating costs the reference is assumed, as by `AddNewKeysToIdentity`. - // Otherwise a key whose hash is already a unique key hash is added without a - // non-unique key hash reference. The probe records no cost operation, so every - // other key gets exactly the operations of `AddNewKeysToIdentity`. - let with_reference_to_non_unique_key = estimated_costs_only_with_layer_info.is_some() - || !self.has_unique_public_key_hash( - key.public_key_hash()?, - transaction, - &platform_version.drive, - )?; - - self.insert_new_non_unique_key_operations( - identity_id, - key, - with_reference_to_non_unique_key, - true, - epoch, - estimated_costs_only_with_layer_info, - transaction, - &mut drive_operations, - platform_version, - )?; - } - Ok(drive_operations) - } } diff --git a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/mod.rs new file mode 100644 index 00000000000..eb33af54b1c --- /dev/null +++ b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/mod.rs @@ -0,0 +1,79 @@ +mod v0; +mod v1; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::estimated_layers::EstimatedLayers; +use dpp::identity::IdentityPublicKey; + +use dpp::block::epoch::Epoch; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; + +impl Drive { + /// The operations for adding new non-unique keys to a masternode identity, version controlled + /// by `add_new_keys_to_masternode_identity`: 0 up to protocol version 14, where it followed + /// `add_new_keys_to_identity`, which is 0 there too. Version 0 emits the operations of + /// `add_new_keys_to_identity_operations` when estimating costs and for every key whose hash is + /// not a unique key hash. + /// + /// A key whose hash is already a unique key hash is added without a non-unique key hash + /// reference, matching how keys are stored when a masternode identity is created. From + /// version 1, a key whose hash the identity already holds a non-unique key hash reference + /// for, or that an earlier key of the same call has, is also added without a reference. The + /// owner, voter and operator identities of masternodes all add keys through this. + /// + /// # Arguments + /// + /// * `identity_id` - The ID of the masternode identity to which keys are to be added. + /// * `non_unique_keys_to_add` - The keys to be added, all of non-unique key types. + /// * `epoch` - The current epoch. + /// * `estimated_costs_only_with_layer_info` - The estimated costs with layer information. + /// * `transaction` - The current transaction. + /// * `platform_version` - The platform version. + /// + /// # Returns + /// + /// * `Result, Error>` - The resulting low level drive operations if successful, or an error. + pub fn add_new_keys_to_masternode_identity_operations( + &self, + identity_id: [u8; 32], + non_unique_keys_to_add: Vec, + epoch: &Epoch, + estimated_costs_only_with_layer_info: &mut Option, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + match platform_version + .drive + .methods + .identity + .update + .add_new_keys_to_masternode_identity + { + 0 => self.add_new_keys_to_masternode_identity_operations_v0( + identity_id, + non_unique_keys_to_add, + epoch, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ), + 1 => self.add_new_keys_to_masternode_identity_operations_v1( + identity_id, + non_unique_keys_to_add, + epoch, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ), + version => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "add_new_keys_to_masternode_identity_operations".to_string(), + known_versions: vec![0, 1], + received: version, + })), + } + } +} diff --git a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v0/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v0/mod.rs new file mode 100644 index 00000000000..4444fd8fd3b --- /dev/null +++ b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v0/mod.rs @@ -0,0 +1,59 @@ +use crate::drive::Drive; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::estimated_layers::EstimatedLayers; +use dpp::identity::identity_public_key::methods::hash::IdentityPublicKeyHashMethodsV0; +use dpp::identity::IdentityPublicKey; + +use dpp::block::epoch::Epoch; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; + +impl Drive { + /// The operations for adding new non-unique keys to a masternode identity + #[inline(always)] + pub(super) fn add_new_keys_to_masternode_identity_operations_v0( + &self, + identity_id: [u8; 32], + non_unique_keys_to_add: Vec, + epoch: &Epoch, + estimated_costs_only_with_layer_info: &mut Option, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let mut drive_operations: Vec = vec![]; + if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_keys_for_identity_id( + identity_id, + estimated_costs_only_with_layer_info, + &platform_version.drive, + )?; + } + + for key in non_unique_keys_to_add { + // When estimating costs the reference is assumed, as by `AddNewKeysToIdentity`. + // Otherwise a key whose hash is already a unique key hash is added without a + // non-unique key hash reference. The probe records no cost operation, so every + // other key gets exactly the operations of `AddNewKeysToIdentity`. + let with_reference_to_non_unique_key = estimated_costs_only_with_layer_info.is_some() + || !self.has_unique_public_key_hash( + key.public_key_hash()?, + transaction, + &platform_version.drive, + )?; + + self.insert_new_non_unique_key_operations( + identity_id, + key, + with_reference_to_non_unique_key, + true, + epoch, + estimated_costs_only_with_layer_info, + transaction, + &mut drive_operations, + platform_version, + )?; + } + Ok(drive_operations) + } +} diff --git a/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v1/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v1/mod.rs new file mode 100644 index 00000000000..70c6c3670f1 --- /dev/null +++ b/packages/rs-drive/src/drive/identity/update/methods/add_new_keys_to_masternode_identity/v1/mod.rs @@ -0,0 +1,82 @@ +use crate::drive::Drive; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::estimated_layers::EstimatedLayers; +use dpp::block::epoch::Epoch; +use dpp::identity::identity_public_key::methods::hash::IdentityPublicKeyHashMethodsV0; +use dpp::identity::IdentityPublicKey; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; +use std::collections::BTreeSet; + +impl Drive { + /// Adds keys to a masternode identity like version 0, except that a key whose public key + /// hash the identity already holds a non-unique key hash reference for, or that an earlier + /// key of the same call has, is added without a second reference. + /// + /// The reference records that an identity holds a key with a hash, once per hash and + /// identity; version 0 fails the batch with `IdentityKeyAlreadyExists` when asked for a + /// second one. A masternode identity's keys come from Core, which accepts any 20 bytes as an + /// operator payout or a platform node id, so an operator identity can need two keys of + /// different kinds with one hash, such as a payout equal to a platform node id it had + /// before. Each key is still stored under its own id with its own purpose. + #[inline(always)] + pub(super) fn add_new_keys_to_masternode_identity_operations_v1( + &self, + identity_id: [u8; 32], + non_unique_keys_to_add: Vec, + epoch: &Epoch, + estimated_costs_only_with_layer_info: &mut Option, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let mut drive_operations: Vec = vec![]; + if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_keys_for_identity_id( + identity_id, + estimated_costs_only_with_layer_info, + &platform_version.drive, + )?; + } + + let mut referenced_hashes = BTreeSet::new(); + for key in non_unique_keys_to_add { + // When estimating costs the reference is assumed, as by version 0, once per hash of + // the call. The probes record no cost operation, as version 0's does not. + let with_reference_to_non_unique_key = if estimated_costs_only_with_layer_info.is_some() + { + referenced_hashes.insert(key.public_key_hash()?) + } else { + let public_key_hash = key.public_key_hash()?; + !self.has_unique_public_key_hash( + public_key_hash, + transaction, + &platform_version.drive, + )? && !self.has_non_unique_public_key_hash_already_for_identity_operations( + public_key_hash, + identity_id, + transaction, + &mut vec![], + &platform_version.drive, + )? + // False for a hash an earlier key of this call references: the batch holds + // that reference already, and a second insert of it would be a duplicate + // operation, which a batch checked for consistency refuses. + && referenced_hashes.insert(public_key_hash) + }; + + self.insert_new_non_unique_key_operations( + identity_id, + key, + with_reference_to_non_unique_key, + true, + epoch, + estimated_costs_only_with_layer_info, + transaction, + &mut drive_operations, + platform_version, + )?; + } + Ok(drive_operations) + } +} diff --git a/packages/rs-drive/src/drive/identity/update/methods/mod.rs b/packages/rs-drive/src/drive/identity/update/methods/mod.rs index c4bccd40633..b10a5206efe 100644 --- a/packages/rs-drive/src/drive/identity/update/methods/mod.rs +++ b/packages/rs-drive/src/drive/identity/update/methods/mod.rs @@ -1,4 +1,5 @@ mod add_new_keys_to_identity; +mod add_new_keys_to_masternode_identity; mod add_new_non_unique_keys_to_identity; mod add_new_unique_keys_to_identity; mod add_to_identity_balance; diff --git a/packages/rs-drive/src/drive/identity/update/mod.rs b/packages/rs-drive/src/drive/identity/update/mod.rs index d6e3dd412b5..de812d128c7 100644 --- a/packages/rs-drive/src/drive/identity/update/mod.rs +++ b/packages/rs-drive/src/drive/identity/update/mod.rs @@ -397,10 +397,16 @@ mod tests { mod add_new_keys_to_masternode_identity { use super::*; + use crate::drive::Drive; + use crate::error::identity::IdentityError; + use crate::error::Error; use crate::util::batch::drive_op_batch::DriveLowLevelOperationConverter; + use crate::util::batch::DriveOperation; use crate::util::batch::IdentityOperationType; use crate::util::estimated_layers::EstimatedLayers; use dpp::block::block_info::BlockInfo; + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dpp::identity::identity_public_key::methods::hash::IdentityPublicKeyHashMethodsV0; use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; use dpp::identity::{IdentityV0, KeyID, KeyType, Purpose, SecurityLevel}; use dpp::platform_value::BinaryData; @@ -489,6 +495,325 @@ mod tests { } } } + + /// A masternode identity holding only a key with hash `[2; 20]`, created like Platform + /// creates one, at `platform_version`. + fn drive_with_masternode_identity(platform_version: &PlatformVersion) -> Drive { + let drive = setup_drive_with_initial_state_structure(Some(platform_version)); + let masternode_identity: Identity = IdentityV0 { + id: MASTERNODE_IDENTITY_ID.into(), + public_keys: BTreeMap::from([( + 0, + masternode_key(0, KeyType::ECDSA_HASH160, Purpose::TRANSFER), + )]), + balance: 0, + revision: 0, + } + .into(); + drive + .add_new_identity( + masternode_identity, + true, + &BlockInfo::default(), + true, + None, + platform_version, + ) + .expect("expected to insert the masternode identity"); + drive + } + + /// A masternode key with id `id`, of `key_type` and `purpose`, and data `data`. + fn key_with_data( + id: KeyID, + key_type: KeyType, + purpose: Purpose, + data: [u8; 20], + ) -> IdentityPublicKey { + IdentityPublicKeyV0 { + id, + key_type, + purpose, + security_level: SecurityLevel::CRITICAL, + read_only: true, + data: BinaryData::new(data.to_vec()), + disabled_at: None, + contract_bounds: None, + } + .into() + } + + fn add_keys( + drive: &Drive, + keys: Vec, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + drive + .apply_drive_operations( + vec![DriveOperation::IdentityOperation( + IdentityOperationType::AddNewKeysToMasternodeIdentity { + identity_id: MASTERNODE_IDENTITY_ID, + non_unique_keys_to_add: keys, + }, + )], + true, + &BlockInfo::default(), + None, + platform_version, + ) + .map(|_| ()) + } + + fn key_ids(drive: &Drive, platform_version: &PlatformVersion) -> Vec { + drive + .fetch_full_identity(MASTERNODE_IDENTITY_ID, None, platform_version) + .expect("expected to fetch the identity") + .expect("expected the identity") + .public_keys() + .keys() + .copied() + .collect() + } + + fn holds_reference(drive: &Drive, hash: [u8; 20]) -> bool { + drive + .has_non_unique_public_key_hash_already_for_identity_operations( + hash, + MASTERNODE_IDENTITY_ID, + None, + &mut vec![], + &PlatformVersion::latest().drive, + ) + .expect("expected to read the reference") + } + + /// Core accepts any 20 bytes as an operator payout or platform node id, so a masternode + /// identity can need a key whose hash a key added to it earlier has, such as a payout + /// equal to an earlier node id. Up to protocol version 14 that key fails the batch with + /// `IdentityKeyAlreadyExists`; from 15 it is stored under its own id, and the identity + /// keeps its one reference for the hash. + #[test] + fn should_add_a_key_whose_hash_the_identity_already_references_from_protocol_version_15() { + let hash = [0x9f; 20]; + for (platform_version, added) in [ + ( + PlatformVersion::get(14).expect("expected protocol version 14"), + false, + ), + (PlatformVersion::latest(), true), + ] { + let drive = drive_with_masternode_identity(platform_version); + add_keys( + &drive, + vec![key_with_data( + 1, + KeyType::EDDSA_25519_HASH160, + Purpose::SYSTEM, + hash, + )], + platform_version, + ) + .expect("expected to add the first key"); + let result = add_keys( + &drive, + vec![key_with_data( + 2, + KeyType::ECDSA_HASH160, + Purpose::TRANSFER, + hash, + )], + platform_version, + ); + let context = format!("protocol version {}", platform_version.protocol_version); + if added { + result.expect("expected to add the second key"); + assert_eq!(key_ids(&drive, platform_version), [0, 1, 2], "{context}"); + } else { + assert!( + matches!( + result, + Err(Error::Identity(IdentityError::IdentityKeyAlreadyExists(_))) + ), + "{context}: {result:?}" + ); + assert_eq!(key_ids(&drive, platform_version), [0, 1], "{context}"); + } + assert!(holds_reference(&drive, hash), "{context}"); + } + } + + /// Two keys with one hash in a single call are both stored from protocol version 15, + /// under one reference for the hash. + #[test] + fn should_add_two_keys_with_one_hash_in_one_call() { + let platform_version = PlatformVersion::latest(); + let hash = [0x9e; 20]; + let drive = drive_with_masternode_identity(platform_version); + add_keys( + &drive, + vec![ + key_with_data(1, KeyType::EDDSA_25519_HASH160, Purpose::SYSTEM, hash), + key_with_data(2, KeyType::ECDSA_HASH160, Purpose::TRANSFER, hash), + ], + platform_version, + ) + .expect("expected to add both keys"); + assert_eq!(key_ids(&drive, platform_version), [0, 1, 2]); + assert!(holds_reference(&drive, hash)); + } + + /// Masternodes whose operators share a payout address add keys of one hash no identity held + /// before, to several identities in one batch. Each identity's operation queues that hash's + /// tree under the non-unique key hashes tree; from protocol version 15 the batch creates it + /// once and keeps every identity's key and entry, also where batches are checked for + /// consistency, which refuse the repeated insert at protocol version 14. + #[test] + fn should_add_keys_of_one_new_hash_to_several_identities_in_one_batch() { + let other_identity_id = [2; 32]; + let hash = [0x9b; 20]; + for (platform_version, applied) in [ + ( + PlatformVersion::get(14).expect("expected protocol version 14"), + false, + ), + (PlatformVersion::latest(), true), + ] { + let drive = drive_with_masternode_identity(platform_version); + let other_identity: Identity = IdentityV0 { + id: other_identity_id.into(), + public_keys: BTreeMap::from([( + 0, + key_with_data(0, KeyType::ECDSA_HASH160, Purpose::TRANSFER, [3; 20]), + )]), + balance: 0, + revision: 0, + } + .into(); + drive + .add_new_identity( + other_identity, + true, + &BlockInfo::default(), + true, + None, + platform_version, + ) + .expect("expected to insert the other masternode identity"); + let add = |identity_id: [u8; 32]| { + DriveOperation::IdentityOperation( + IdentityOperationType::AddNewKeysToMasternodeIdentity { + identity_id, + non_unique_keys_to_add: vec![key_with_data( + 1, + KeyType::ECDSA_HASH160, + Purpose::TRANSFER, + hash, + )], + }, + ) + }; + let result = drive.apply_drive_operations( + vec![add(MASTERNODE_IDENTITY_ID), add(other_identity_id)], + true, + &BlockInfo::default(), + None, + platform_version, + ); + let context = format!("protocol version {}", platform_version.protocol_version); + if applied { + result.expect("expected to add both keys"); + assert_eq!(key_ids(&drive, platform_version), [0, 1], "{context}"); + assert!(holds_reference(&drive, hash), "{context}"); + assert!( + drive + .has_non_unique_public_key_hash_already_for_identity_operations( + hash, + other_identity_id, + None, + &mut vec![], + &platform_version.drive, + ) + .expect("expected to read the reference"), + "{context}" + ); + } else { + assert!(result.is_err(), "{context}: {result:?}"); + } + } + } + + /// Estimating the cost of a call with two keys of one hash counts the hash's reference + /// once, as applying it writes it once: fewer operations than for two keys of two hashes. + #[test] + fn should_estimate_one_reference_for_two_keys_with_one_hash() { + let platform_version = PlatformVersion::latest(); + let drive = drive_with_masternode_identity(platform_version); + let estimated_operations = |second_hash: [u8; 20]| { + let mut estimated_costs = Some(EstimatedLayers::new()); + IdentityOperationType::AddNewKeysToMasternodeIdentity { + identity_id: MASTERNODE_IDENTITY_ID, + non_unique_keys_to_add: vec![ + key_with_data(1, KeyType::EDDSA_25519_HASH160, Purpose::SYSTEM, [0x9d; 20]), + key_with_data(2, KeyType::ECDSA_HASH160, Purpose::TRANSFER, second_hash), + ], + } + .into_low_level_drive_operations( + &drive, + &mut estimated_costs, + &BlockInfo::default(), + None, + platform_version, + ) + .expect("expected operations") + .len() + }; + assert!(estimated_operations([0x9d; 20]) < estimated_operations([0x9c; 20])); + } + + /// A key whose hash is another identity's unique key hash is added without a non-unique + /// key hash reference, at protocol version 14 and from 15 alike. + #[test] + fn should_add_a_key_with_a_unique_key_hash_without_a_reference() { + for platform_version in [ + PlatformVersion::get(14).expect("expected protocol version 14"), + PlatformVersion::latest(), + ] { + let drive = drive_with_masternode_identity(platform_version); + let user_identity = Identity::random_identity(3, Some(15), platform_version) + .expect("expected a random identity"); + let unique_hash = user_identity + .public_keys() + .values() + .find(|key| key.key_type().is_unique_key_type()) + .expect("expected a unique key") + .public_key_hash() + .expect("expected the key's hash"); + drive + .add_new_identity( + user_identity, + false, + &BlockInfo::default(), + true, + None, + platform_version, + ) + .expect("expected to insert the user identity"); + add_keys( + &drive, + vec![key_with_data( + 1, + KeyType::ECDSA_HASH160, + Purpose::TRANSFER, + unique_hash, + )], + platform_version, + ) + .expect("expected to add the key"); + let context = format!("protocol version {}", platform_version.protocol_version); + assert_eq!(key_ids(&drive, platform_version), [0, 1], "{context}"); + assert!(!holds_reference(&drive, unique_hash), "{context}"); + } + } } mod disable_identity_keys { diff --git a/packages/rs-drive/src/util/batch/drive_op_batch/drive_methods/apply_drive_operations/v2/mod.rs b/packages/rs-drive/src/util/batch/drive_op_batch/drive_methods/apply_drive_operations/v2/mod.rs index 3b7b69f625c..b89c31e376e 100644 --- a/packages/rs-drive/src/util/batch/drive_op_batch/drive_methods/apply_drive_operations/v2/mod.rs +++ b/packages/rs-drive/src/util/batch/drive_op_batch/drive_methods/apply_drive_operations/v2/mod.rs @@ -12,13 +12,14 @@ use grovedb::TransactionArg; use dpp::version::PlatformVersion; -use crate::drive::RootTree; +use crate::drive::{non_unique_key_hashes_tree_path_vec, RootTree}; use crate::util::batch::drive_op_batch::finalize_task::{ DriveOperationFinalizationTasks, DriveOperationFinalizeTask, }; use crate::util::estimated_layers::EstimatedLayers; use grovedb::batch::key_info::KeyInfo::KnownKey; use grovedb::batch::KeyInfoPath; +use std::collections::HashSet; /// The level of the root tree each root key sits on, as `drive/mod.rs` draws the root tree and /// `grovedb-structure.json` records it (`layer_shapes.root`, kept in step with a real GroveDB by @@ -92,7 +93,9 @@ impl Drive { /// descriptions name a shallower one (the credit shielded pool's, at level 3, for /// `ShieldedBalances` at level 4). The estimate does not depend on the order of the batch's operations, and no /// write is priced at a shallower depth than its own description. Applying a batch is - /// generation 1's. + /// generation 1's, except that the batch creates a hash's tree under the non-unique key hashes + /// tree once where several of its identity operations each queue it + /// (`retain_one_insert_per_non_unique_key_hash_tree`). #[inline(always)] pub(crate) fn apply_drive_operations_v2( &self, @@ -207,6 +210,8 @@ impl Drive { .map(|layers| layers.resolve(platform_version)) .transpose()?; + retain_one_insert_per_non_unique_key_hash_tree(&mut low_level_operations); + let repaid_identity_debt = LowLevelDriveOperation::take_repaid_identity_debt(&mut low_level_operations)?; @@ -288,6 +293,27 @@ impl Drive { } } +/// Keeps the first of the identical inserts that create one hash's tree under the non-unique key +/// hashes tree, and drops the others. +/// +/// Each identity operation is lowered on its own, against stored state. Several identities adding +/// a key of one hash no identity held before, in one batch (masternodes whose operators share a +/// payout address, say), therefore each queue the same tree, beside their own entries under it. +/// One insert creates it; a batch checked for consistency refuses the repeats. +fn retain_one_insert_per_non_unique_key_hash_tree(operations: &mut Vec) { + let hashes_tree_path = + KeyInfoPath::from_known_owned_path(non_unique_key_hashes_tree_path_vec()); + let mut inserted = HashSet::new(); + operations.retain(|operation| match operation { + LowLevelDriveOperation::GroveOperation(grove_operation) + if grove_operation.path == hashes_tree_path => + { + inserted.insert(grove_operation.clone()) + } + _ => true, + }); +} + #[cfg(test)] mod tests { use super::super::tests::{deep, estimate, mint, shallow, system_credits}; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v11.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v11.rs index ffa5923ec3e..ff844d7623a 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v11.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_method_versions/v11.rs @@ -113,6 +113,16 @@ const DKG_MINING_SCHEDULES_V1: &[DkgMiningSchedule] = &[ /// only safe together with this check. /// - `engine.consensus_params_update` 2 installs the full evidence policy when it changes. /// +/// The operator payouts: `core_based_updates.update_masternode_list` is 2 rather than 1, which +/// keeps the operator payout and the revived height of an updated masternode's Core diff, reads +/// from Core (`protx info` at the block's chain locked height) the payout of an operator key +/// change Core prints without one, and reads whether Core's V24 is active from the record +/// `update_core_info` 1 writes before it; and +/// `core_based_updates.masternode_updates.update_operator_identity` is 1 rather than 0, which +/// reconciles every TRANSFER key of the operator identity it updates with the payout. The two only +/// work together: version 0 adds a second key with the data of a payout Core re-sends, which fails +/// the block. +/// /// The asset unlocks: `withdrawals.build_untied_withdrawal_transactions_from_documents` is 1 /// rather than 0: a withdrawal pooled once `update_core_info` 1 has recorded Dash Core's V24 as /// active gets asset unlock payload version 2, whose txid every re-signed instance shares, @@ -134,7 +144,9 @@ pub const DRIVE_ABCI_METHOD_VERSIONS_V11: DriveAbciMethodVersions = DriveAbciMet }, core_based_updates: DriveAbciCoreBasedUpdatesMethodVersions { update_core_info: 1, // changed in v15: records the chain locked Core height at which V24 is first seen active + update_masternode_list: 2, // changed in v15: keeps a diff's operator payouts and revived heights, reads from Core the payout of an operator key change printed without one, reads V24 from the Core fork record masternode_updates: DriveAbciMasternodeIdentitiesUpdatesMethodVersions { + update_operator_identity: 1, // changed in v15: reconciles the TRANSFER keys of the identity it updates with the operator payout, without fallbacks update_voter_identity: 1, // changed in v15: re-enables the voter key of the address a masternode returns to ..DRIVE_ABCI_METHOD_VERSIONS_V10 .core_based_updates diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs index c55c74dd04e..3508c5dbe81 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/mod.rs @@ -238,6 +238,10 @@ pub struct DriveIdentityUpdateMethodVersions { pub add_new_non_unique_keys_to_identity: FeatureVersion, pub add_new_unique_keys_to_identity: FeatureVersion, pub add_new_keys_to_identity: FeatureVersion, + /// Adds keys to a masternode identity. Version 1 (protocol version 15) adds a key whose hash + /// the identity already holds a non-unique key hash reference for, or that an earlier key of + /// the same call has, without a second reference. + pub add_new_keys_to_masternode_identity: FeatureVersion, pub insert_identity_balance: FeatureVersion, pub initialize_negative_identity_balance: FeatureVersion, pub add_to_identity_balance: FeatureVersion, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs index fd26e1af65b..e7315d6baad 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v1.rs @@ -112,6 +112,7 @@ pub const DRIVE_IDENTITY_METHOD_VERSIONS_V1: DriveIdentityMethodVersions = add_new_non_unique_keys_to_identity: 0, add_new_unique_keys_to_identity: 0, add_new_keys_to_identity: 0, + add_new_keys_to_masternode_identity: 0, insert_identity_balance: 0, initialize_negative_identity_balance: 0, add_to_identity_balance: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs index 360c20b0661..e8ddd72534c 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_identity_method_versions/v2.rs @@ -169,6 +169,7 @@ pub const DRIVE_IDENTITY_METHOD_VERSIONS_V2: DriveIdentityMethodVersions = add_new_non_unique_keys_to_identity: 0, add_new_unique_keys_to_identity: 0, add_new_keys_to_identity: 0, + add_new_keys_to_masternode_identity: 0, insert_identity_balance: 0, initialize_negative_identity_balance: 0, add_to_identity_balance: 1, // changed: credits that repay an identity's debt are owed to the processing fee pool diff --git a/packages/rs-platform-version/src/version/drive_versions/v10.rs b/packages/rs-platform-version/src/version/drive_versions/v10.rs index 19cf3009495..d1e0d80fbae 100644 --- a/packages/rs-platform-version/src/version/drive_versions/v10.rs +++ b/packages/rs-platform-version/src/version/drive_versions/v10.rs @@ -6,6 +6,7 @@ use crate::version::drive_versions::drive_document_method_versions::v5::DRIVE_DO use crate::version::drive_versions::drive_identity_method_versions::v2::DRIVE_IDENTITY_METHOD_VERSIONS_V2; use crate::version::drive_versions::drive_identity_method_versions::{ DriveIdentityCostEstimationMethodVersions, DriveIdentityMethodVersions, + DriveIdentityUpdateMethodVersions, }; use crate::version::drive_versions::drive_state_transition_method_versions::v5::DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V5; use crate::version::drive_versions::drive_token_method_versions::v3::DRIVE_TOKEN_METHOD_VERSIONS_V3; @@ -81,6 +82,15 @@ use grovedb_version::version::v5::GROVE_V5; /// * **Core fork activations**: `platform_system.fetch_core_fork_activation_height` and /// `store_core_fork_activation_height` are `Some(0)` (`None` before): Drive keeps, in `Misc`, /// the chain locked Core height at which Platform first saw each tracked Core fork active. +/// * **Masternode keys that repeat a hash**: `identity.update.add_new_keys_to_masternode_identity` +/// is 1 rather than 0: a key added to a masternode identity whose public key hash the identity +/// already holds a non-unique key hash reference for, or that an earlier key of the same batch +/// has, gets no second reference instead of failing with `IdentityKeyAlreadyExists`. Core +/// accepts any 20 bytes as an operator payout or platform node id, so an operator identity can +/// need two keys of different kinds with one hash. The keys Platform adds to owner and voter +/// identities go through the same method. `apply_drive_operations` 2 also creates a hash's +/// tree under the non-unique key hashes tree once where several identity operations of one +/// batch each queue it, as masternodes whose operators share a payout address do. /// * **Refunds at the storing epoch's rate** (protocol v15 note 27): `fees.calculate_fee` 1 /// prices the bytes a refund returns at the storage rate of the protocol version that ran the /// epoch that stored them, which that epoch's tree records. @@ -111,6 +121,10 @@ pub const DRIVE_VERSION_V10: DriveVersion = DriveVersion { for_token_balances: 1, // changed in v10: keeps a deeper root and Tokens layer description ..DRIVE_IDENTITY_METHOD_VERSIONS_V2.cost_estimation }, + update: DriveIdentityUpdateMethodVersions { + add_new_keys_to_masternode_identity: 1, // changed in v10: a masternode key whose hash the identity already holds a reference for is added without a second one + ..DRIVE_IDENTITY_METHOD_VERSIONS_V2.update + }, ..DRIVE_IDENTITY_METHOD_VERSIONS_V2 }, document: DRIVE_DOCUMENT_METHOD_VERSIONS_V5, // changed in v10: private document type stores, delete estimates diff --git a/packages/rs-platform-version/src/version/v15.rs b/packages/rs-platform-version/src/version/v15.rs index aa794085509..efe898683d5 100644 --- a/packages/rs-platform-version/src/version/v15.rs +++ b/packages/rs-platform-version/src/version/v15.rs @@ -567,12 +567,44 @@ pub const PROTOCOL_VERSION_15: ProtocolVersion = 15; /// block's rate. Every schedule so far has the same storage rate, so no refund changes. A /// removal from an earlier epoch pays once per such epoch for reading its record, the only /// fee this changes; generation 0 reads nothing. +/// +/// 28. **Operator payouts follow Dash Core**: `update_masternode_list` 2 +/// (`DRIVE_ABCI_METHOD_VERSIONS_V11`) keeps the operator payout of an updated masternode's +/// Core diff, where earlier versions took operator payouts only from full masternode states, +/// so a payout set or changed after Platform first listed a masternode never reached it. +/// Core empties the payout on every operator key change and prints a payout only when it +/// differs between the diff's two Core heights, so for a key change printed without one the +/// payout is read from Core's state of that masternode at the block's chain locked height +/// (`protx info`, one call for each operator key change or revocation of a listed +/// masternode that the diff prints without a payout): the new operator no longer inherits the previous operator's payout key, nor +/// starts without the one it set. A Core error fails the block. An operator key Core only +/// re-encodes is still the stored key, and whether Core's V24 is active is read from the +/// Core fork record (note 18) instead of asked of Core. A diff that does not print the PoSe +/// revived height keeps the stored one. `update_operator_identity` 1 reconciles every +/// TRANSFER key of the operator identity it updates with the payout: the payout's key is +/// kept, re-enabled or added, and every other enabled TRANSFER key disabled, with no +/// fallback to an earlier operator's key, so the same payout applied twice changes nothing +/// the second time. A key change never touches the TRANSFER keys of an identity that already +/// exists, whatever the payout: the identity a masternode leaves keeps them, and so does an +/// identity it switches back to, until that identity's next payout change. Each step of +/// `update_operator_identity` 1 touches only its own kind of key, and a key whose hash +/// another key of the identity has is added like any other: `DRIVE_VERSION_V10`'s +/// `add_new_keys_to_masternode_identity` 1 adds it without a second non-unique key hash +/// reference where version 0 failed the block. To fix what earlier versions left, the first +/// block of this version (`transition_to_version_15`) reads Dash Core's full masternode list +/// at the last committed chain locked core height, the height Platform's list is at then. +/// Where a stored operator payout differs from Core's, it stores Core's, and it reconciles +/// the TRANSFER keys of every masternode's current operator identity with Core's payout +/// through `update_operator_identity` 1. A Core error, or a Core list whose masternodes are +/// not Platform's, fails that block. Protocol v14 and earlier keep +/// `update_masternode_list` 0 or 1, `update_operator_identity` 0 and +/// `add_new_keys_to_masternode_identity` 0. pub const PLATFORM_V15: PlatformVersion = PlatformVersion { protocol_version: PROTOCOL_VERSION_15, - drive: DRIVE_VERSION_V10, // changed: calculate_fee 1 prices a refund at the storing epoch's rate (note 27); apply_contract_with_serialization 1 writes nothing for a contract whose format version 1 bytes match the stored one's (note 8); vote method versions v4: register_identity_vote 1 registers a yes/no vote, remove_all_votes_given_by_identities 1 also removes the yes/no votes of a masternode that left the list, every yes_no method and remove_vote_poll_end_date_query_operations Some(0) make yes/no polls active, add_vote_poll_end_date_query_operations 1 adds only the entry when the batch already queues the end date's tree with other storage flags, and verify methods v4 make the yes/no proof verifiers active (note 10) and verify_private_document_entries Some(0) a page of a private type's store (note 13); state transition method versions v5: document_create_transition 1 deletes a consumed document from what its lookup read (note 11); document method versions v5: insert_empty_private_document_store_operation Some(0) creates a private document type's store, and fetch_private_document_entries and prove_private_document_entries Some(0) read and prove a page of it (note 13); apply_drive_operations 2 resolves a batch's estimate, remove_reference_for_index_level_for_contract_operations 2, stateless_delete_of_non_tree_for_costs 1 and for_total_system_credits_update 1 describe their layers as they are (note 14); create_initial_state_structure 5 adds the token marketplace escrow trees; calculate_total_tokens_balance 2 counts escrowed tokens; the escrow primitives; the identity balance, token balance, prefunded specialized balance and contract fee pot estimations (for_balances 1, for_token_balances 1, estimated_cost_for_prefunded_specialized_balance_update 2, add_estimation_costs_for_contract_fee_pot_update 1) keep a deeper layer description (note 17); fetch_core_fork_activation_height and store_core_fork_activation_height Some(0) keep the Core fork activations (note 18); document method versions v5: append_private_document_entries_operations Some(0) appends a private document create's entries (note 25); grove_version GROVE_V5 (note 26) + drive: DRIVE_VERSION_V10, // changed: calculate_fee 1 prices a refund at the storing epoch's rate (note 27); apply_contract_with_serialization 1 writes nothing for a contract whose format version 1 bytes match the stored one's (note 8); vote method versions v4: register_identity_vote 1 registers a yes/no vote, remove_all_votes_given_by_identities 1 also removes the yes/no votes of a masternode that left the list, every yes_no method and remove_vote_poll_end_date_query_operations Some(0) make yes/no polls active, add_vote_poll_end_date_query_operations 1 adds only the entry when the batch already queues the end date's tree with other storage flags, and verify methods v4 make the yes/no proof verifiers active (note 10) and verify_private_document_entries Some(0) a page of a private type's store (note 13); state transition method versions v5: document_create_transition 1 deletes a consumed document from what its lookup read (note 11); document method versions v5: insert_empty_private_document_store_operation Some(0) creates a private document type's store, and fetch_private_document_entries and prove_private_document_entries Some(0) read and prove a page of it (note 13); apply_drive_operations 2 resolves a batch's estimate, remove_reference_for_index_level_for_contract_operations 2, stateless_delete_of_non_tree_for_costs 1 and for_total_system_credits_update 1 describe their layers as they are (note 14); create_initial_state_structure 5 adds the token marketplace escrow trees; calculate_total_tokens_balance 2 counts escrowed tokens; the escrow primitives; the identity balance, token balance, prefunded specialized balance and contract fee pot estimations (for_balances 1, for_token_balances 1, estimated_cost_for_prefunded_specialized_balance_update 2, add_estimation_costs_for_contract_fee_pot_update 1) keep a deeper layer description (note 17); fetch_core_fork_activation_height and store_core_fork_activation_height Some(0) keep the Core fork activations (note 18); document method versions v5: append_private_document_entries_operations Some(0) appends a private document create's entries (note 25); grove_version GROVE_V5 (note 26); add_new_keys_to_masternode_identity 1 adds a masternode key whose hash the identity already holds a reference for (note 28) drive_abci: DriveAbciVersion { structs: DRIVE_ABCI_STRUCTURE_VERSIONS_V2, - methods: DRIVE_ABCI_METHOD_VERSIONS_V11, // changed: the yes/no poll's tally, record, clean-up and finish hook are Some(0) (note 10); verify_chain_lock_locally 1 rejects a chain lock locally only when Platform holds the quorums active at its signing height, and verify_chain_lock_through_core 1 has Core's verifychainlock check the others before submitchainlock; consensus_params_update 2 installs the full evidence policy (note 16); update_core_info 1 records the chain locked height at which Core's V24 is first seen active (note 18); build_untied_withdrawal_transactions_from_documents 1 emits asset unlock payload version 2 once V24 is recorded as active (note 20); update_voter_identity 1 re-enables the voter key of the address a masternode returns to (note 22) + methods: DRIVE_ABCI_METHOD_VERSIONS_V11, // changed: the yes/no poll's tally, record, clean-up and finish hook are Some(0) (note 10); verify_chain_lock_locally 1 rejects a chain lock locally only when Platform holds the quorums active at its signing height, and verify_chain_lock_through_core 1 has Core's verifychainlock check the others before submitchainlock; consensus_params_update 2 installs the full evidence policy (note 16); update_core_info 1 records the chain locked height at which Core's V24 is first seen active (note 18); build_untied_withdrawal_transactions_from_documents 1 emits asset unlock payload version 2 once V24 is recorded as active (note 20); update_voter_identity 1 re-enables the voter key of the address a masternode returns to (note 22); update_masternode_list 2 and update_operator_identity 1 make operator payouts and the operator identities' TRANSFER keys follow Core's diffs (note 28) validation_and_processing: DRIVE_ABCI_VALIDATION_VERSIONS_V11, // changed: identity update advanced_structure 1 refuses unpaid an identity update whose added key's proof of possession does not verify (note 2); create_domain_data_trigger 2 (note 3); contract update transform_into_action 1 holds an update to the compiled size cap in check_tx (note 8); batch advanced_structure 2 refuses a token config update on a read-only contract (note 9); masternode vote transform_into_action 1 looks up a masternode's previous answer on a yes/no poll (note 10); document_reference_validation 1 keeps what a consumed commitment's lookup read (note 11); document create structure_validation 2 holds a decision proposal to its fund, rule and end date (note 12); data trigger bindings 3 refuse a replace or a delete of a token market order (note 17) and drop the DashPay bindings (note 24); document create state_validation 3 keeps a create outside a contest off the values of a live one its index values do not decide (note 19); identity create basic_structure 1 checks the identity id before the asset lock proof and advanced_structure 2 no longer does (note 23) withdrawal_constants: DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V3, query: DRIVE_ABCI_QUERY_VERSIONS_V2,