diff --git a/packages/rs-platform-wallet-ffi/ERROR_CODE_REGISTRY.md b/packages/rs-platform-wallet-ffi/ERROR_CODE_REGISTRY.md index 211a7d709ca..b1cda68de23 100644 --- a/packages/rs-platform-wallet-ffi/ERROR_CODE_REGISTRY.md +++ b/packages/rs-platform-wallet-ffi/ERROR_CODE_REGISTRY.md @@ -114,7 +114,7 @@ These are shipped ABI. Do not renumber. | 98 | `NotFound` | Sentinel — `Option` returned as an error | | 99 | `ErrorUnknown` | Sentinel — unmapped/flattened errors | -**Next allocatable integer: 61** — 27–60 are all claimed (27, 29, 31, 34–42 +**Next allocatable integer: 74**: 27–73 are all claimed (27, 29, 31, 34–42 and 46 merged; 43–45 proposed by active #4313 at head `0302b188ab`; 47 and 48 proposed by active #4356 (47 renumbered from 42, 48 from 43 — see their rows below); 49–54 proposed by active #4586 (the persister @@ -123,7 +123,11 @@ shield debits and durable recovery errors; 58 proposed by #4799 for an unavailab identity balance response; 59 proposed by #5322 for an SPV teardown that needs a process restart; 60 proposed by the pooled Core-to-shielded wallet drain for `ErrorAssetLockTooManyInputs` (Rust, Swift and Kotlin mirrors — -renumbered from 59 when #5322 took that number on `v5.1-dev`); 28, 30, +renumbered from 59 when #5322 took that number on `v5.1-dev`); 61–67 held +by open branches with no row here yet (61 `ErrorAssetLockRecipientMismatch`, +62 both `ErrorShieldedClaimBindingMismatch` and `ErrorIdentityRevisionMismatch`, +63–66 the invitation-claim preflight, 67 `ErrorShieldedTreeInconsistent`); 68 +to 73 proposed for DashPay private contact requests; 28, 30, 32 and 33 reserved). **28, 30, 32 and 33 are RESERVED, not free**: 28 and 30 were vacated when the reservation trio moved to 34–36; 32 and 33 lapsed when their in-repo owners @@ -177,6 +181,12 @@ Fork-era numbers remain in the collision history, which is immutable record. | 58 | `ErrorIdentityBalanceUnavailable` | #4799 | Proposed — Platform returned no balance for a managed identity. Retrying the read is safe; this is distinct from missing wallet ownership and must not trigger registration or funding | | 59 | `ErrorSpvProcessRestartRequired` | #5322 | Proposed — SPV startup or teardown panicked; restart the host process. Not retryable, unlike 27, which SPV start, stop and storage clear now return while an SPV teardown is still running. Typed in Rust, Swift and Kotlin (the PR also types 27 on Kotlin) | | 60 | `ErrorAssetLockTooManyInputs` | Core-to-shielded pooled Max | Proposed — an asset lock needs more inputs than one may carry (100 on Dash Core relay policy and Platform consensus): the full drain, or since #5712 a fixed amount held only in many small coins (identity registration, top-up, invitation, platform-address funding). Two meanings. On a build: nothing was broadcast, the reservation is released, no partial drain, and a rebuild after consolidating is safe. On the resume of a tracked `Built` / `Broadcast` lock over the cap (since #5714): it is not re-sent, but an earlier attempt may have reached peers; the row stays tracked and nothing is released. Rust, Swift and Kotlin retain the typed refusal. Renumbered from 59 after #5322 claimed it | +| 68 | `ErrorPrivateContactRequestNeedsPrivateReply` | DashPay private contact requests (FFI and Swift) | Proposed. The contact requests with that identity came or went privately (protocol version 15), so a `contactRequest` document to it, in reply or as a rotation, would publish the relationship: the public accept refuses a reply, and the public send refuses a request or a rotation. Nothing was sent; the host accepts with `platform_wallet_manager_accept_contact_request_replying_in_kind` (which picks the reply itself, so a host calling it does not route on 68) or `platform_wallet_manager_accept_private_contact_request`, or sends with `platform_wallet_manager_send_private_contact_request`. Taken above 61–67, which open branches already hold. Rust and Swift (raw case, `init(ffi:)` arm, typed `PlatformWalletError.privateContactRequestNeedsPrivateReply` and an `ErrorHandlingTests` pin) carry it; Kotlin falls through to `PlatformWallet.Generic`, which keeps the integer | +| 69 | `ErrorShieldedBuildRequired` | DashPay private contact requests (FFI, Swift and Kotlin) | Proposed. The operation needs the shielded pool and the library was built without the `shielded` feature. Today only `platform_wallet_manager_accept_contact_request_replying_in_kind`, exported in every build, returns it, for a request that came privately: its private reply is paid from the pool. Nothing was sent; a public or adopted reply works on such a build. Not retryable on the same build. No existing code meant "shielded unsupported" (`ErrorWalletOperation` with a not-configured message is a runtime state of a shielded build), so it takes the next free number. Swift (raw case, `init(ffi:)` arm, typed `PlatformWalletError.shieldedBuildRequired`, `ErrorHandlingTests` pin) and Kotlin (`DashSdkError.PlatformWallet.ShieldedBuildRequired`, `DashSdkErrorTest` pin) type it | +| 70 | `ErrorProtocolVersionNotActive` | DashPay private contact requests (FFI, Swift and Kotlin) | Proposed. The SDK refused a request without sending it (`dash_sdk::Error::ProtocolVersionNotActive`), because the network is proven at a protocol version below the one that serves it or the SDK is pinned below it: for example the private contact request sync on a network before protocol version 15. Expected on such a network, so a host may stay quiet about it; it lifts when the network upgrades. Mapped by the blanket conversion for every operation that propagates the SDK error. A refusal that may lift sooner (`is_transient_protocol_version_refusal`: the network's version could not be confirmed) keeps the catch-all `ErrorUnknown`, as before. Previously both fell to `ErrorUnknown`. Swift (`PlatformWalletError.protocolVersionNotActive`) and Kotlin (`DashSdkError.PlatformWallet.ProtocolVersionNotActive`) type it | +| 71 | `ErrorPrivateContactRequestPending` | DashPay private contact requests (FFI, Swift and Kotlin) | Proposed. Maps `PlatformWalletError::PrivateContactRequestPending`: a private contact request to that identity is in flight (another private send or reply reserved it before sealing), or was paid for and a short read-back sync did not find the entry in the store yet; or the identity's private store could not be read to its end before its first private send or accept since load, which the wallet reads first so that a reply paid for in an earlier process is adopted rather than paid for again. Nothing new was paid; a later private contact request sync, or a retry, settles it. Pending, not failed: the host must not submit it again. Returned by the private send, the private accept and the in-kind accept; the shielded spend contract (`map_spend_result`) keeps the code rather than flattening it to `ErrorWalletOperation`. Swift (`PlatformWalletError.privateContactRequestPending`, `ErrorHandlingTests` pin) and Kotlin (`DashSdkError.PlatformWallet.PrivateContactRequestPending`, `DashSdkErrorTest` pin) type it, and both example apps keep the submission pending | +| 72 | `ErrorPrivateContactPaymentsNotRestored` | DashPay private contact requests (FFI, Swift and Kotlin) | Proposed. Maps `PlatformWalletError::PrivateContactPaymentsNotRestored`: wallet readiness, not one contact's pending request (71). The wallet was loaded after binding a shielded account in an earlier process, or the identity was loaded with its private scan state marking a private payment that may be unresolved, so its unconfirmed private payments may hold pairs no reservation shows yet. The public send, the public accept and the in-kind accept are refused with nothing signed or broadcast until they are restored: configuring the shielded store restores them with no keys, as binding the wallet's shielded account does; a private sync does not, and a build without the `shielded` feature never does. `platform_wallet_get_private_contact_readiness` reads the state, so a host shows it rather than inferring it. Swift and Kotlin must type it, as for 71, and treat it as held until the readiness reads restored | +| 73 | `ErrorPrivateContactRecoveryUnsupported` | DashPay private contact requests (FFI, Swift and Kotlin) | Proposed. Maps `PlatformWalletError::PrivateContactRecoveryUnsupported`: the persister does not declare `PRIVATE_CONTACT_REQUEST_SCAN_STATE` (bit 14) together with `ATOMIC_CHANGESETS` (bit 0, with the changeset begin and end callbacks wired), so the private send, the private accept and the in-kind accept's private reply are refused before anything is sealed or paid. The payment's recovery marker could not survive a restart, and a process reopening the wallet without the `shielded` feature, or before the shielded store is configured, could publish the pair while the payment may still execute. Not retryable as it is: a host that persists the private contact request scan state in changesets it commits whole declares both. Swift and Kotlin must type it | **Code 31 left this table on 2026-08-04.** `ErrorSigningKeyUnavailable` sat here as #4183's proposal until #4183 merged (`189a3abb1c`); it is now in the merged diff --git a/packages/rs-platform-wallet-ffi/src/contact_request.rs b/packages/rs-platform-wallet-ffi/src/contact_request.rs index 433adc5d014..217e36c4651 100644 --- a/packages/rs-platform-wallet-ffi/src/contact_request.rs +++ b/packages/rs-platform-wallet-ffi/src/contact_request.rs @@ -4,9 +4,10 @@ use crate::error::*; use crate::handle::*; +use crate::runtime::block_on_worker; use crate::types::*; use crate::{check_ptr, unwrap_option_or_return, unwrap_result_or_return}; -use platform_wallet::ContactRequest; +use platform_wallet::{ContactRequest, PrivateContactReadiness}; // Storage for contact requests lazy_static::lazy_static! { @@ -226,11 +227,236 @@ pub extern "C" fn contact_request_destroy(request_handle: Handle) -> PlatformWal } } +/// What a contact request from one of a wallet's identities waits for, as the +/// wallet decides it (`platform_wallet::PrivateContactReadiness`), for a host +/// to show rather than infer from the refusals it got. +#[repr(C)] +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct PrivateContactReadinessFFI { + /// Whether the wallet's private payments from an earlier process are + /// restored for the identity. While false, its public sends and accepts, + /// the in-kind accept included, are refused with + /// `ErrorPrivateContactPaymentsNotRestored` (72) until the shielded store + /// is configured or the wallet's shielded account bound; no private sync + /// lifts it. A build without the `shielded` feature never does. + pub private_payments_restored: bool, + /// Whether a private sync read DashPay's private contact request store to + /// its end for the identity since load. Until then its first send or + /// accept reads it first, and is refused with + /// `ErrorPrivateContactRequestPending` (71) while that read cannot finish. + pub private_store_read: bool, + /// Whether a private contact request the identity paid for may still be + /// unresolved: the marker its private scan state stores. A contact's own + /// pending request is `managed_identity_get_private_reply_pending`. + pub unresolved_private_payment: bool, +} + +impl From for PrivateContactReadinessFFI { + fn from(readiness: PrivateContactReadiness) -> Self { + Self { + private_payments_restored: readiness.private_payments_restored, + private_store_read: readiness.private_store_read, + unresolved_private_payment: readiness.unresolved_private_payment, + } + } +} + +/// What a contact request from `identity_id`, one of the wallet's identities, +/// waits for, as the wallet decides it, into `out_readiness` +/// ([`PrivateContactReadinessFFI`]): so a host tells "restoring private +/// payments" from "reading the private store" and from a contact's pending +/// request by the wallet's own state. A pure read of live wallet state. +/// +/// `out_readiness` is reset to all false first. An unknown `wallet_handle` is +/// `NotFound`; an identity the wallet does not hold is refused as its lookup +/// is. +/// +/// # Safety +/// - `identity_id` must point to 32 readable bytes. +/// - `out_readiness` must be a valid, writable pointer. +#[no_mangle] +pub unsafe extern "C" fn platform_wallet_get_private_contact_readiness( + wallet_handle: Handle, + identity_id: *const u8, + out_readiness: *mut PrivateContactReadinessFFI, +) -> PlatformWalletFFIResult { + check_ptr!(out_readiness); + unsafe { *out_readiness = PrivateContactReadinessFFI::default() }; + let identity_id = unwrap_result_or_return!(unsafe { read_identifier(identity_id) }); + let option = PLATFORM_WALLET_STORAGE.with_item(wallet_handle, |wallet| { + let identity = wallet.identity().clone(); + block_on_worker(async move { + identity + .dashpay() + .private_contact_readiness(&identity_id) + .await + }) + }); + let result = unwrap_option_or_return!(option); + let readiness = unwrap_result_or_return!(result); + unsafe { *out_readiness = readiness.into() }; + PlatformWalletFFIResult::ok() +} + #[cfg(test)] mod tests { use super::*; + + /// Each readiness flag reaches its own FFI field, alone: a conversion + /// that swapped or cleared one would show here. + #[test] + fn should_convert_each_readiness_flag_to_its_own_field() { + let restored = PrivateContactReadinessFFI::from(PrivateContactReadiness { + private_payments_restored: true, + private_store_read: false, + unresolved_private_payment: false, + }); + assert_eq!( + restored, + PrivateContactReadinessFFI { + private_payments_restored: true, + private_store_read: false, + unresolved_private_payment: false, + } + ); + let read = PrivateContactReadinessFFI::from(PrivateContactReadiness { + private_payments_restored: false, + private_store_read: true, + unresolved_private_payment: false, + }); + assert_eq!( + read, + PrivateContactReadinessFFI { + private_payments_restored: false, + private_store_read: true, + unresolved_private_payment: false, + } + ); + let unresolved = PrivateContactReadinessFFI::from(PrivateContactReadiness { + private_payments_restored: false, + private_store_read: false, + unresolved_private_payment: true, + }); + assert_eq!( + unresolved, + PrivateContactReadinessFFI { + private_payments_restored: false, + private_store_read: false, + unresolved_private_payment: true, + } + ); + } use dpp::prelude::Identifier; + /// The readiness of a fresh wallet's identity: nothing held, the store not + /// read yet, no private payment. A null out pointer, an unknown wallet and + /// an identity the wallet does not hold are refused, with the out value + /// reset. + #[test] + fn should_read_a_wallet_identity_s_private_contact_readiness() { + use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; + use dpp::identity::v0::IdentityV0; + use dpp::identity::{Identity, IdentityPublicKey, KeyType, Purpose, SecurityLevel}; + use dpp::platform_value::BinaryData; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + + assert_eq!( + unsafe { + platform_wallet_get_private_contact_readiness( + 1, + [0u8; 32].as_ptr(), + std::ptr::null_mut(), + ) + } + .code, + PlatformWalletFFIResultCode::ErrorNullPointer + ); + let mut readiness = PrivateContactReadinessFFI { + private_payments_restored: true, + private_store_read: true, + unresolved_private_payment: true, + }; + let unknown = unsafe { + platform_wallet_get_private_contact_readiness( + Handle::MAX, + [0u8; 32].as_ptr(), + &mut readiness, + ) + }; + assert_eq!(unknown.code, PlatformWalletFFIResultCode::NotFound); + assert_eq!(readiness, PrivateContactReadinessFFI::default(), "reset"); + + let manager = crate::test_fixtures::mock_manager(); + let wallet = crate::runtime::runtime() + .block_on(manager.create_wallet_from_seed_bytes( + key_wallet::Network::Testnet, + &[9; 64], + WalletAccountCreationOptions::Default, + Some(0), + )) + .expect("a wallet"); + let identity_id = Identifier::from([0xB2; 32]); + let key = dpp::secp256k1::PublicKey::from_secret_key( + &dpp::secp256k1::SecretKey::from_secret_bytes([4; 32]).expect("a secret key"), + ); + let identity = Identity::V0(IdentityV0 { + id: identity_id, + public_keys: [( + 1, + IdentityPublicKey::V0(IdentityPublicKeyV0 { + id: 1, + purpose: Purpose::ENCRYPTION, + security_level: SecurityLevel::MEDIUM, + contract_bounds: None, + key_type: KeyType::ECDSA_SECP256K1, + read_only: false, + data: BinaryData::new(key.serialize().to_vec()), + disabled_at: None, + }), + )] + .into_iter() + .collect(), + balance: 0, + revision: 0, + }); + crate::runtime::runtime().block_on(async { + let mut wm = wallet.wallet_manager().write().await; + wm.get_wallet_info_mut(&wallet.wallet_id()) + .expect("the wallet's info") + .identity_manager + .add_identity(identity, 0, wallet.wallet_id(), wallet.persister()) + .expect("the identity"); + }); + let handle = PLATFORM_WALLET_STORAGE.insert(wallet); + + let read = unsafe { + platform_wallet_get_private_contact_readiness( + handle, + identity_id.to_buffer().as_ptr(), + &mut readiness, + ) + }; + assert_eq!(read.code, PlatformWalletFFIResultCode::Success); + assert_eq!( + readiness, + PrivateContactReadinessFFI { + private_payments_restored: true, + private_store_read: false, + unresolved_private_payment: false, + } + ); + let stranger = unsafe { + platform_wallet_get_private_contact_readiness( + handle, + [0xEE; 32].as_ptr(), + &mut readiness, + ) + }; + assert_ne!(stranger.code, PlatformWalletFFIResultCode::Success); + assert_eq!(readiness, PrivateContactReadinessFFI::default(), "reset"); + PLATFORM_WALLET_STORAGE.remove(handle); + } + #[test] fn test_contact_request_getters() { unsafe { diff --git a/packages/rs-platform-wallet-ffi/src/dashpay.rs b/packages/rs-platform-wallet-ffi/src/dashpay.rs index 85dde7ca6ea..fc04b02c647 100644 --- a/packages/rs-platform-wallet-ffi/src/dashpay.rs +++ b/packages/rs-platform-wallet-ffi/src/dashpay.rs @@ -35,8 +35,19 @@ use std::ffi::CStr; use std::os::raw::c_char; -use platform_wallet::ContactRequest; -use rs_sdk_ffi::{MnemonicResolverCoreSigner, MnemonicResolverHandle, SignerHandle, VTableSigner}; +use dash_sdk::platform::dashpay::PrivateContactRequest; +use dash_sdk::platform::private_document_entries::PrivateDocumentEntries; +use dash_sdk::platform::private_document_scan::PrivateDocumentScan; +use dpp::identity::{Identity, KeyID}; +use dpp::platform_value::BinaryData; +use dpp::prelude::{DataContract, Identifier}; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; +use platform_wallet::{ContactRequest, PlatformWalletError, PrivateDocumentKeyPaths}; +use rs_sdk_ffi::{ + MnemonicResolverCoreSigner, MnemonicResolverHandle, MnemonicResolverSignerError, SignerHandle, + VTableSigner, +}; use crate::contact_request::CONTACT_REQUEST_STORAGE; use crate::core_wallet::classify_broadcast_result; @@ -1106,10 +1117,12 @@ impl platform_wallet::ContactCryptoProvider for ResolverContactCryptoProvider { path: &key_wallet::bip32::DerivationPath, ) -> Result { use key_wallet::signer::ExtendedPubKeySigner; + // A missing mnemonic stays the key-unavailable outcome: the private sync's seed check + // derives through here before any page is read. self.signer .extended_public_key(path) .await - .map_err(|e| platform_wallet::PlatformWalletError::InvalidIdentityData(e.to_string())) + .map_err(private_document_signer_error) } async fn ecdh_shared_secret( @@ -1222,6 +1235,95 @@ impl platform_wallet::ContactCryptoProvider for ResolverContactCryptoProvider { private_data: opened.private_data, }) } + + async fn scan_private_document_page( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + start_position: u64, + page: &PrivateDocumentEntries, + ) -> Result { + self.signer + .scan_private_document_page( + &keys.recipients, + &keys.senders, + data_contract_id, + document_type_name, + start_position, + page, + ) + .map_err(private_document_signer_error) + } + + async fn scan_private_document_pages( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + pages: &[(u64, &PrivateDocumentEntries)], + ) -> Result, PlatformWalletError> { + // One Keychain read for the batch, not one per page. + self.signer + .scan_private_document_pages( + &keys.recipients, + &keys.senders, + data_contract_id, + document_type_name, + pages, + ) + .map_err(private_document_signer_error) + } + + async fn seal_private_contact_request( + &self, + sender_encryption_key_path: &key_wallet::bip32::DerivationPath, + dashpay_contract: &DataContract, + sender: &Identity, + sender_encryption_key_id: KeyID, + signing_key_id: KeyID, + signature: &[u8; 65], + recipient: &Identity, + recipient_key_id: KeyID, + request: &PrivateContactRequest, + esk_seed: &[u8; 32], + platform_version: &PlatformVersion, + ) -> Result { + self.signer + .seal_private_contact_request( + sender_encryption_key_path, + dashpay_contract, + sender, + sender_encryption_key_id, + signing_key_id, + signature, + recipient, + recipient_key_id, + request, + esk_seed, + platform_version, + ) + .map_err(private_document_signer_error) + } +} + +/// A signer failure of a private document operation, or of the receiving xpub +/// derivation the private sync's seed check makes first, as a wallet error: the +/// SDK's refusal stays typed; a missing mnemonic is the key-unavailable +/// outcome (FFI code 31), carried as the signer's typed completion is, with +/// the reserved marker at position 0 of a `ProtocolError::Generic` payload +/// (`NotFound` renders it there), so the structural discriminator restores +/// it; the rest are reported as the other provider methods report theirs. +fn private_document_signer_error(error: MnemonicResolverSignerError) -> PlatformWalletError { + match error { + MnemonicResolverSignerError::PrivateContactRequest(error) => { + PlatformWalletError::Sdk(error) + } + error @ MnemonicResolverSignerError::NotFound => PlatformWalletError::Sdk( + dash_sdk::Error::Protocol(ProtocolError::Generic(error.to_string())), + ), + error => PlatformWalletError::InvalidIdentityData(error.to_string()), + } } /// Drain the persisted deferred-crypto queue using the Keychain signer for the @@ -1653,6 +1755,228 @@ pub unsafe extern "C" fn platform_wallet_drainable_contact_crypto_count( mod tests { use super::*; + /// A test wallet's mnemonic, counting how often the resolver reads it. + unsafe extern "C" fn counting_resolve( + ctx: *const std::ffi::c_void, + _wallet_id_bytes: *const u8, + out_buf: *mut c_char, + out_capacity: usize, + out_len: *mut usize, + ) -> i32 { + const PHRASE: &str = + "abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon \ + abandon about"; + (*(ctx as *const std::sync::atomic::AtomicUsize)) + .fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let phrase = PHRASE.as_bytes(); + if phrase.len() + 1 > out_capacity { + return rs_sdk_ffi::mnemonic_resolver_result::BUFFER_TOO_SMALL; + } + std::ptr::copy_nonoverlapping(phrase.as_ptr() as *const c_char, out_buf, phrase.len()); + *out_buf.add(phrase.len()) = 0; + *out_len = phrase.len(); + rs_sdk_ffi::mnemonic_resolver_result::SUCCESS + } + + unsafe extern "C" fn missing_resolve( + _ctx: *const std::ffi::c_void, + _wallet_id_bytes: *const u8, + _out_buf: *mut c_char, + _out_capacity: usize, + _out_len: *mut usize, + ) -> i32 { + rs_sdk_ffi::mnemonic_resolver_result::NOT_FOUND + } + + unsafe extern "C" fn noop_destroy(_ctx: *mut std::ffi::c_void) {} + + /// A page of entries no key of the wallet opens. + fn store_page(start: u64, len: u64, total_count: u64) -> PrivateDocumentEntries { + PrivateDocumentEntries { + chunk_power: None, + entries: (start..start + len) + .map(|position| { + dash_sdk::platform::private_document_entries::PrivateDocumentEntry { + position, + block_height: position, + block_time_ms: position, + data: vec![position as u8; 589], + } + }) + .collect(), + total_count, + } + } + + /// The resolver-backed provider scans DashPay's private store, which a + /// public send or accept reads first: a page, and a batch of pages with + /// one Keychain read for the batch. A missing mnemonic is the + /// key-unavailable error, never an empty page. + #[test] + fn should_scan_a_page_and_a_batch_with_one_resolve_each() { + use platform_wallet::ContactCryptoProvider; + use std::str::FromStr; + + let path = + |path: &str| key_wallet::bip32::DerivationPath::from_str(path).expect("a valid path"); + let identity_id = Identifier::from([0xB0; 32]); + let keys = PrivateDocumentKeyPaths { + recipients: vec![(identity_id, vec![(2, path("m/9'/1'/5'/0'/0'/1'/2'"))])], + senders: vec![(identity_id, 1, path("m/9'/1'/5'/0'/0'/0'/1'"))], + }; + let contract_id = Identifier::from([0x44; 32]); + let resolves = std::sync::atomic::AtomicUsize::new(0); + let resolver = unsafe { + rs_sdk_ffi::dash_sdk_mnemonic_resolver_create( + &resolves as *const std::sync::atomic::AtomicUsize as *mut std::ffi::c_void, + counting_resolve, + noop_destroy, + ) + }; + let provider = unsafe { + resolver_contact_crypto_provider(resolver, [0u8; 32], key_wallet::Network::Testnet) + }; + let reads = || resolves.load(std::sync::atomic::Ordering::SeqCst); + let runtime = tokio::runtime::Builder::new_current_thread() + .build() + .expect("a runtime"); + + let first = store_page(0, 4, 6); + let page = runtime + .block_on(provider.scan_private_document_page( + &keys, + contract_id, + "privateContactRequest", + 0, + &first, + )) + .expect("the provider scans a page"); + assert_eq!(reads(), 1); + assert!(page.is_of_store(&contract_id, "privateContactRequest")); + assert_eq!(page.next_position, 4); + assert!(page.received.is_empty() && page.sent.is_empty() && page.rejected.is_empty()); + + let second = store_page(4, 2, 6); + let batch = runtime + .block_on(provider.scan_private_document_pages( + &keys, + contract_id, + "privateContactRequest", + &[(0, &first), (4, &second)], + )) + .expect("the provider scans a batch"); + assert_eq!(reads(), 2, "one Keychain read for the batch"); + assert_eq!( + batch + .iter() + .map(|scan| (scan.next_position, scan.is_complete())) + .collect::>(), + vec![(4, false), (6, true)] + ); + unsafe { rs_sdk_ffi::dash_sdk_mnemonic_resolver_destroy(resolver) }; + + let missing = unsafe { + rs_sdk_ffi::dash_sdk_mnemonic_resolver_create( + std::ptr::null_mut(), + missing_resolve, + noop_destroy, + ) + }; + let provider = unsafe { + resolver_contact_crypto_provider(missing, [0u8; 32], key_wallet::Network::Testnet) + }; + let refused = runtime.block_on(provider.scan_private_document_pages( + &keys, + contract_id, + "privateContactRequest", + &[(0, &first)], + )); + assert!( + matches!(refused, Err(PlatformWalletError::Sdk(_))), + "the key is unavailable: {refused:?}" + ); + unsafe { rs_sdk_ffi::dash_sdk_mnemonic_resolver_destroy(missing) }; + } + + /// A private sync whose resolver has no mnemonic is refused at its seed check, before any + /// page is read, with the key-unavailable code (31), as the page scan is, so a host routes + /// it to key repair rather than reporting an unknown failure. + #[test] + fn should_report_a_missing_mnemonic_at_the_private_sync_seed_check_as_key_unavailable() { + use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; + use dpp::identity::v0::IdentityV0; + use dpp::identity::{IdentityPublicKey, KeyType, Purpose, SecurityLevel}; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + + let manager = crate::test_fixtures::mock_manager(); + let wallet = runtime() + .block_on(manager.create_wallet_from_seed_bytes( + key_wallet::Network::Testnet, + &[7; 64], + WalletAccountCreationOptions::Default, + Some(0), + )) + .expect("a wallet"); + let key = dpp::secp256k1::PublicKey::from_secret_key( + &dpp::secp256k1::SecretKey::from_secret_bytes([3; 32]).expect("a secret key"), + ); + let identity = Identity::V0(IdentityV0 { + id: Identifier::from([0xB1; 32]), + public_keys: [( + 1, + IdentityPublicKey::V0(IdentityPublicKeyV0 { + id: 1, + purpose: Purpose::ENCRYPTION, + security_level: SecurityLevel::MEDIUM, + contract_bounds: None, + key_type: KeyType::ECDSA_SECP256K1, + read_only: false, + data: BinaryData::new(key.serialize().to_vec()), + disabled_at: None, + }), + )] + .into_iter() + .collect(), + balance: 0, + revision: 0, + }); + runtime().block_on(async { + let mut wm = wallet.wallet_manager().write().await; + wm.get_wallet_info_mut(&wallet.wallet_id()) + .expect("the wallet's info") + .identity_manager + .add_identity(identity, 0, wallet.wallet_id(), wallet.persister()) + .expect("the identity"); + }); + + let missing = unsafe { + rs_sdk_ffi::dash_sdk_mnemonic_resolver_create( + std::ptr::null_mut(), + missing_resolve, + noop_destroy, + ) + }; + let provider = unsafe { + resolver_contact_crypto_provider( + missing, + wallet.wallet_id(), + key_wallet::Network::Testnet, + ) + }; + let refused = runtime().block_on( + wallet + .identity() + .dashpay() + .sync_private_contact_requests(&provider, 1), + ); + let error = refused.expect_err("the mnemonic is missing"); + assert_eq!( + PlatformWalletFFIResult::from(error).code, + PlatformWalletFFIResultCode::ErrorSigningKeyUnavailable + ); + unsafe { rs_sdk_ffi::dash_sdk_mnemonic_resolver_destroy(missing) }; + } + // Marshalling-boundary coverage for the verify entry point, replacing what // the removed attach FFI's input-validation tests upheld. The crypto // semantics (matching seed binds, wrong seed rejected) are pinned library- diff --git a/packages/rs-platform-wallet-ffi/src/error.rs b/packages/rs-platform-wallet-ffi/src/error.rs index 762a8e35668..ab7f47a7aff 100644 --- a/packages/rs-platform-wallet-ffi/src/error.rs +++ b/packages/rs-platform-wallet-ffi/src/error.rs @@ -136,9 +136,14 @@ pub enum PlatformWalletFFIResultCode { ErrorWalletAlreadyExists = 15, /// Maps `PlatformWalletError::ShieldedBroadcastFailed`. The shielded /// transition (identity-create, unshield, transfer, or withdrawal) was - /// DEFINITIVELY not executed — either the relay/CheckTx rejected the - /// broadcast, or Platform reported the transition's own execution error. - /// Any note reservations were released and the caller is free to retry. + /// DEFINITIVELY not executed. For a shielded spend that consumes notes, it + /// never left the wallet: it was refused before anything was sent (no + /// address to send it to, or the SDK's own refusal). An endpoint's verdict + /// proves nothing for such a spend, as no proof backs it: that is + /// [`Self::ErrorShieldedSpendUnconfirmed`], as is any broadcast that may + /// have been delivered. A shield into the pool also reports here a CheckTx + /// or execution verdict. Any note reservations were released and the + /// caller is free to retry. /// For identity-create, the new identity does NOT exist and /// `out_identity_id` is left untouched (still zeroed). ErrorShieldedBroadcastFailed = 16, @@ -152,14 +157,18 @@ pub enum PlatformWalletFFIResultCode { /// the caller can hold the slot and surface the pending identity. ErrorShieldedBroadcastUnconfirmed = 17, /// Maps `PlatformWalletError::ShieldedSpendUnconfirmed` (unshield / - /// shielded transfer / shielded withdrawal). The spend transition was - /// ACCEPTED by the relay but its execution result could not be confirmed - /// (DAPI wait timeout, result-proof fetch/verify failure, …). The spend - /// may have executed on chain, so the wallet intentionally KEEPS the - /// notes reserved: the next nullifier sync promotes them to spent if the - /// spend landed, and an app restart frees them if it never did. The host - /// must NOT auto-retry — a retry would select different unreserved notes - /// and could double-send if the original spend landed. + /// shielded transfer / shielded withdrawal / identity top-up and create / + /// private document create from the pool), and + /// `PlatformWalletError::ShieldedBroadcastMaybeDelivered`. The spend + /// transition was accepted by the relay, or may have been delivered, but + /// its execution result could not be confirmed (a broadcast attempt that + /// failed in transport, a DAPI wait timeout, a result-proof fetch/verify + /// failure, …). The spend may have executed on chain, so the wallet + /// intentionally KEEPS the notes reserved and its recovery record: the + /// next nullifier sync promotes them to spent if the spend landed, and the + /// redrive and anchor-prune passes release them once it provably never + /// can. The host must NOT auto-retry — a retry would select different + /// unreserved notes and could double-send if the original spend landed. ErrorShieldedSpendUnconfirmed = 18, /// Maps `PlatformWalletError::ShieldedNoRecordedAnchor`. The wallet could /// not build the spend against any Platform-recorded anchor yet: its local @@ -622,6 +631,72 @@ pub enum PlatformWalletFFIResultCode { /// coins lets either lock fit. ErrorAssetLockTooManyInputs = 60, + /// Maps `PlatformWalletError::PrivateContactRequestNeedsPrivateReply`: + /// the contact requests with that identity came or went privately + /// (protocol version 15), so a public `contactRequest` to it, in reply or + /// as a rotation, would publish the relationship. The public send and the + /// public accept refuse it, and nothing was sent. Send it with + /// `platform_wallet_manager_send_private_contact_request`, or accept it + /// with `platform_wallet_manager_accept_contact_request_replying_in_kind` + /// (which replies privately itself) or + /// `platform_wallet_manager_accept_private_contact_request`, instead. + /// 61 to 67 are claimed by open branches, so this takes 68. + ErrorPrivateContactRequestNeedsPrivateReply = 68, + + /// The operation needs the shielded pool, and this library was built + /// without the `shielded` feature: today, accepting a contact request + /// that came privately with + /// `platform_wallet_manager_accept_contact_request_replying_in_kind`, + /// whose private reply is paid from the pool. Nothing was sent. Not + /// retryable on this build; a build with shielded support accepts it. + ErrorShieldedBuildRequired = 69, + + /// The SDK refused the request without sending it: the network is proven + /// at a protocol version below the one that serves it (for example + /// DashPay's private contact request store before protocol version 15), + /// or the SDK is pinned below it. Expected on such a network; it lifts + /// when the network upgrades. A refusal whose query for the network's + /// version failed keeps the catch-all code, whether that may pass or comes + /// from the SDK's own setup (a context provider that cannot give quorum + /// keys at all), as neither shows the network runs an earlier version. + ErrorProtocolVersionNotActive = 70, + + /// Maps `PlatformWalletError::PrivateContactRequestPending`: a private + /// contact request to that identity is in flight (another send or reply + /// reserved it), or was paid for and is not yet read back from the store; + /// or this identity's private store could not be read to its end before + /// its first private send or accept since load (the wallet reads it first, + /// so a reply paid for in an earlier process is adopted rather than paid + /// for again). Nothing new was paid. Do not submit it again: a later + /// private contact request sync + /// (`platform_wallet_sync_private_contact_requests`) or a retry settles + /// it. + ErrorPrivateContactRequestPending = 71, + + /// Maps `PlatformWalletError::PrivateContactPaymentsNotRestored`: the + /// wallet, not one contact, is held. It was loaded after binding a + /// shielded account in an earlier process, or the identity was loaded with + /// its private scan state marking a private payment that may be + /// unresolved, so its unconfirmed private payments may hold pairs no + /// reservation shows yet: no public contact request is sent or accepted, + /// nothing signed or broadcast, until they are restored from the shielded + /// store. Configuring the shielded store restores them with no keys; so + /// does binding the wallet's shielded account. A private sync does not. + /// `platform_wallet_get_private_contact_readiness` reads whether they are + /// restored. A build without the `shielded` feature never restores them. + ErrorPrivateContactPaymentsNotRestored = 72, + + /// Maps `PlatformWalletError::PrivateContactRecoveryUnsupported`: the + /// persister does not declare that it keeps the private contact request + /// scan state (`PRIVATE_CONTACT_REQUEST_SCAN_STATE`, bit 14) in atomic + /// changesets (`ATOMIC_CHANGESETS`, bit 0, with the changeset begin and end + /// callbacks wired), so a private contact request is not paid for, nothing + /// sealed or paid. Its recovery + /// marker could not survive a restart, and a process reopening the wallet + /// could then publish the pair while the payment may still execute. Not + /// retryable as it is: a host persisting the scan state declares both. + ErrorPrivateContactRecoveryUnsupported = 73, + /// The named thing does not exist. /// /// Originally (and still mostly) the code for every `Option` returned as an @@ -916,7 +991,11 @@ impl From for PlatformWalletFFIResult { PlatformWalletError::ShieldedBroadcastUnconfirmed { .. } => { PlatformWalletFFIResultCode::ErrorShieldedBroadcastUnconfirmed } - PlatformWalletError::ShieldedSpendUnconfirmed { .. } => { + // A broadcast that may have been delivered may still execute: it + // reads as unconfirmed, never as failed, so a host does not pay + // again with other notes. + PlatformWalletError::ShieldedSpendUnconfirmed { .. } + | PlatformWalletError::ShieldedBroadcastMaybeDelivered(..) => { PlatformWalletFFIResultCode::ErrorShieldedSpendUnconfirmed } PlatformWalletError::ShieldedIdentityDebitPending { .. } => { @@ -1059,6 +1138,27 @@ impl From for PlatformWalletFFIResult { PlatformWalletError::SpvProcessRestartRequired(..) => { PlatformWalletFFIResultCode::ErrorSpvProcessRestartRequired } + // The public send or accept refused a contact whose requests came + // or went privately; the host routes it to the private send or + // accept. + PlatformWalletError::PrivateContactRequestNeedsPrivateReply(..) => { + PlatformWalletFFIResultCode::ErrorPrivateContactRequestNeedsPrivateReply + } + // A private request to the contact is in flight or paid and not + // yet read back: pending, not failed, so the host must not submit + // it again. + PlatformWalletError::PrivateContactRequestPending(..) => { + PlatformWalletFFIResultCode::ErrorPrivateContactRequestPending + } + // The wallet's private payments from an earlier process are not + // restored yet: wallet readiness, not one contact's pending request. + PlatformWalletError::PrivateContactPaymentsNotRestored => { + PlatformWalletFFIResultCode::ErrorPrivateContactPaymentsNotRestored + } + // The persister cannot keep a private payment's recovery marker. + PlatformWalletError::PrivateContactRecoveryUnsupported => { + PlatformWalletFFIResultCode::ErrorPrivateContactRecoveryUnsupported + } // A signer failure can also reach this blanket impl wrapped as // `PlatformWalletError::Sdk(dash_sdk::Error::Protocol(..))` (any // wallet operation that propagates the SDK error via `?`). The @@ -1168,6 +1268,20 @@ impl From for PlatformWalletFFIResult { // `MessageSigningKeyUnavailable` mapped above. By the time a // `MessageSigningFailed` reason exists, any marker in it sits // mid-string and is deliberately not matched. + // The network does not serve the request yet, as its proven + // protocol version shows, or the SDK is pinned below it. A + // refusal whose query for the version failed stays on the + // catch-all, whether that may pass (the network failing) or not + // (a context provider configured so it cannot give quorum keys: + // this SDK's own setup), as neither says the network runs an + // earlier version. + PlatformWalletError::Sdk(dash_sdk::Error::ProtocolVersionNotActive { + cause, + pinned, + .. + }) if *pinned || cause.is_none() => { + PlatformWalletFFIResultCode::ErrorProtocolVersionNotActive + } _ => PlatformWalletFFIResultCode::ErrorUnknown, }; let result = PlatformWalletFFIResult::err(code, error.to_string()); @@ -2692,3 +2806,151 @@ mod identity_balance_error_tests { assert!(!result.message.is_null()); } } + +#[cfg(test)] +mod private_contact_request_error_tests { + use super::*; + #[cfg(feature = "shielded")] + use crate::shielded_send::map_spend_result; + + /// A store the network does not serve yet (DashPay's private contact + /// requests before protocol version 15) is told apart from a failure, so + /// a host can stay quiet about it; a refusal that may lift soon, because + /// the version could not be confirmed, is still the network failing. + #[test] + fn should_tell_a_store_the_network_does_not_serve_yet_by_its_own_code() { + let refused = |cause: Option>| { + PlatformWalletFFIResult::from(PlatformWalletError::Sdk( + dash_sdk::Error::ProtocolVersionNotActive { + method: "get_private_document_entries", + required: 15, + feature: None, + network: 14, + pinned: false, + cause, + }, + )) + }; + let lasting = refused(None); + assert_eq!( + lasting.code, + PlatformWalletFFIResultCode::ErrorProtocolVersionNotActive + ); + assert_eq!(lasting.code as u32, 70); + + let transient = refused(Some(Box::new(dash_sdk::Error::Generic( + "no node answered".to_string(), + )))); + assert_eq!(transient.code, PlatformWalletFFIResultCode::ErrorUnknown); + + // A provider that cannot give quorum keys at all is this SDK's setup, + // not the network's version: the catch-all too. + let misconfigured = refused(Some(Box::new(dash_sdk::Error::ContextProviderError( + dash_sdk::error::ContextProviderError::Config("no quorum source".to_string()), + )))); + assert_eq!( + misconfigured.code, + PlatformWalletFFIResultCode::ErrorUnknown + ); + } + + #[test] + fn should_tell_a_private_request_from_other_accept_failures_by_code() { + let result = PlatformWalletFFIResult::from( + PlatformWalletError::PrivateContactRequestNeedsPrivateReply([7; 32].into()), + ); + assert_eq!( + result.code, + PlatformWalletFFIResultCode::ErrorPrivateContactRequestNeedsPrivateReply + ); + assert_eq!(result.code as u32, 68); + assert!(!result.message.is_null()); + } + + /// A private request in flight, or paid and not yet read back, is pending + /// rather than failed, whichever path reports it: the blanket conversion, + /// and the shielded spend contract the private send and accepts use. + #[test] + fn should_report_a_pending_private_request_by_its_own_code() { + let pending = || PlatformWalletError::PrivateContactRequestPending([7; 32].into()); + let result = PlatformWalletFFIResult::from(pending()); + assert_eq!( + result.code, + PlatformWalletFFIResultCode::ErrorPrivateContactRequestPending + ); + assert_eq!(result.code as u32, 71); + #[cfg(feature = "shielded")] + assert_eq!( + map_spend_result(Err(pending()), "private contact request").code, + PlatformWalletFFIResultCode::ErrorPrivateContactRequestPending + ); + } + + /// A public contact request a loaded wallet refuses until its private + /// payments are restored reaches the host by its own code, not as a + /// contact's pending request. + #[test] + fn should_report_unrestored_private_payments_by_their_own_code() { + let result = + PlatformWalletFFIResult::from(PlatformWalletError::PrivateContactPaymentsNotRestored); + assert_eq!( + result.code, + PlatformWalletFFIResultCode::ErrorPrivateContactPaymentsNotRestored + ); + assert_eq!(result.code as u32, 72); + #[cfg(feature = "shielded")] + assert_eq!( + map_spend_result( + Err(PlatformWalletError::PrivateContactPaymentsNotRestored), + "private contact request" + ) + .code, + PlatformWalletFFIResultCode::ErrorPrivateContactPaymentsNotRestored + ); + } + + /// A private payment over a persister that cannot keep its recovery marker + /// reaches the host by its own code, as does a spend whose broadcast may + /// have been delivered by the broadcast-failed one. + #[test] + fn should_report_unsupported_private_recovery_by_its_own_code() { + let result = + PlatformWalletFFIResult::from(PlatformWalletError::PrivateContactRecoveryUnsupported); + assert_eq!( + result.code, + PlatformWalletFFIResultCode::ErrorPrivateContactRecoveryUnsupported + ); + assert_eq!(result.code as u32, 73); + // A broadcast that may have been delivered reads as unconfirmed. + let delivered = PlatformWalletFFIResult::from( + PlatformWalletError::ShieldedBroadcastMaybeDelivered("unavailable".to_string()), + ); + assert_eq!( + delivered.code, + PlatformWalletFFIResultCode::ErrorShieldedSpendUnconfirmed + ); + // The explicit private send and accept report them through the spend + // contract alike. + #[cfg(feature = "shielded")] + { + assert_eq!( + map_spend_result( + Err(PlatformWalletError::PrivateContactRecoveryUnsupported), + "private contact request" + ) + .code, + PlatformWalletFFIResultCode::ErrorPrivateContactRecoveryUnsupported + ); + assert_eq!( + map_spend_result( + Err(PlatformWalletError::ShieldedBroadcastMaybeDelivered( + "unavailable".to_string() + )), + "private contact request" + ) + .code, + PlatformWalletFFIResultCode::ErrorShieldedSpendUnconfirmed + ); + } + } +} diff --git a/packages/rs-platform-wallet-ffi/src/shielded_send.rs b/packages/rs-platform-wallet-ffi/src/shielded_send.rs index 232d8a9a9c8..dd280e0914b 100644 --- a/packages/rs-platform-wallet-ffi/src/shielded_send.rs +++ b/packages/rs-platform-wallet-ffi/src/shielded_send.rs @@ -596,7 +596,7 @@ fn poke_sync_on_unconfirmed(result: &Result, handle: /// withdraw) to a typed FFI result, mirroring the identity-create sibling's /// code split so hosts can tell "definitively failed, safe to retry" from /// "may have executed, do NOT retry". -fn map_spend_result( +pub(crate) fn map_spend_result( result: Result<(), PlatformWalletError>, operation: &str, ) -> PlatformWalletFFIResult { @@ -608,7 +608,11 @@ fn map_spend_result( // shield reserves nothing. Either way the next sync (or an app // restart) reconciles the outcome; the typed Display already // carries the operation name and guidance. - Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => { + // A broadcast that may have been delivered reads the same: it may + // still execute, so its notes and recovery record stay, and a host + // must not pay again. + Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + | Err(e @ PlatformWalletError::ShieldedBroadcastMaybeDelivered(_)) => { PlatformWalletFFIResult::err( PlatformWalletFFIResultCode::ErrorShieldedSpendUnconfirmed, e.to_string(), @@ -620,6 +624,23 @@ fn map_spend_result( Err(e @ PlatformWalletError::ShieldedIdentityDebitPending { .. }) | Err(e @ PlatformWalletError::ShieldedRecoveryCorrupted { .. }) | Err(e @ PlatformWalletError::ShieldedRecoveryKeysRequired { .. }) => e.into(), + // A private contact request in flight, or paid and not yet read back + // from the store: nothing new was paid, and a later private sync + // settles it, so it keeps its own pending code; unrestored private + // payments keep theirs, and so does a persister that cannot keep a + // private payment's recovery marker. A wallet removed before the + // payment started keeps the lifecycle refusal code (`NotFound`) every + // other export gives it. A protocol-version refusal from the store + // read before any payment keeps the blanket conversion's split: a + // lasting refusal is `ErrorProtocolVersionNotActive`, a transient one + // the catch-all. + Err(e @ PlatformWalletError::PrivateContactRequestPending(..)) + | Err(e @ PlatformWalletError::PrivateContactPaymentsNotRestored) + | Err(e @ PlatformWalletError::PrivateContactRecoveryUnsupported) + | Err(e @ PlatformWalletError::WalletRemovedBeforeSend(..)) + | Err(e @ PlatformWalletError::Sdk(dash_sdk::Error::ProtocolVersionNotActive { .. })) => { + e.into() + } // Retryable: the wallet couldn't build the spend against any // Platform-recorded anchor yet (its commitment tree is mid-block after // an index-chunk sync). Nothing was broadcast and the notes were @@ -628,8 +649,9 @@ fn map_spend_result( PlatformWalletFFIResultCode::ErrorShieldedNoRecordedAnchor, format!("Wallet is still syncing to a confirmed state — try again shortly. ({e})"), ), - // Definitive failure: the transition was not executed and the notes - // were released; the host may retry. + // Definitive failure: the transition never left the wallet (for a note + // spend, refused before anything was sent); the notes were released + // and the host may retry. Err(e @ PlatformWalletError::ShieldedBroadcastFailed(_)) => PlatformWalletFFIResult::err( PlatformWalletFFIResultCode::ErrorShieldedBroadcastFailed, format!("{operation} failed: {e}"), @@ -2425,6 +2447,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: true, + anchor_height: None, }, ) .expect("seed malformed durable metadata"); @@ -2992,6 +3015,19 @@ mod tests { assert!(message_of(&result).contains("this request was not started")); } + /// A private payment refused because its wallet was removed first keeps + /// the lifecycle refusal code, not the generic operation failure. + #[test] + fn map_spend_result_keeps_a_wallet_removed_before_send_at_not_found() { + let removed: Result<(), PlatformWalletError> = Err( + PlatformWalletError::WalletRemovedBeforeSend("wallet removed".to_string()), + ); + let result = map_spend_result(removed, "private contact request"); + assert_eq!(result.code, PlatformWalletFFIResultCode::NotFound); + assert_eq!(result.code as i32, 98); + assert!(message_of(&result).contains("wallet removed")); + } + /// A key-unavailable signer completion preserved under `Sdk` by the wallet /// layer must keep code 31 through `map_spend_result` (the shield from /// identity path) instead of flattening to `ErrorWalletOperation`; a diff --git a/packages/rs-platform-wallet/Cargo.toml b/packages/rs-platform-wallet/Cargo.toml index 7da2bb5ce6c..ff178bd57f1 100644 --- a/packages/rs-platform-wallet/Cargo.toml +++ b/packages/rs-platform-wallet/Cargo.toml @@ -61,10 +61,11 @@ grovedb-commitment-tree = { workspace = true, optional = true } # is built against (its `Cargo.lock` resolves rusqlite at this major). rusqlite = { version = "0.38", optional = true } zip32 = { version = "0.2.0", default-features = false, optional = true } -# `StreamExt`/`pin_mut!` to consume the SDK's `sync_shielded_notes_stream` -# in the interleaved fetch/tree-append consumer (shielded sync only). -# Same version as `dash-sdk` so the lockfile resolves a single copy. -futures = { version = "0.3.30", optional = true } +# `StreamExt` to consume the SDK's page streams: the shielded note sync's +# `sync_shielded_notes_stream` and the private contact request sync's +# `private_document_pages`. Same version as `dash-sdk` so the lockfile +# resolves a single copy. +futures = { version = "0.3.30" } # Networked, opt-in example binaries. Each one performs real network I/O # against a live devnet, so they are examples (compiled, never run by @@ -129,7 +130,7 @@ default = ["bls", "eddsa"] test-utils = ["key-wallet/test-utils"] bls = ["key-wallet/bls", "key-wallet-manager/bls"] eddsa = ["key-wallet/eddsa", "key-wallet-manager/eddsa"] -shielded = ["dep:grovedb-commitment-tree", "dep:rusqlite", "dep:zip32", "dep:futures", "dash-sdk/shielded", "dpp/shielded-client"] +shielded = ["dep:grovedb-commitment-tree", "dep:rusqlite", "dep:zip32", "dash-sdk/shielded", "dpp/shielded-client"] # Opt-in serde derives on the changeset types in `src/changeset/` plus # the per-identity / DashPay scalar types those changesets carry. # Activates `key-wallet/serde` (which transitively activates diff --git a/packages/rs-platform-wallet/README.md b/packages/rs-platform-wallet/README.md index 1fc475bed45..f893ac6b246 100644 --- a/packages/rs-platform-wallet/README.md +++ b/packages/rs-platform-wallet/README.md @@ -121,8 +121,35 @@ Recovery preserves the original signed record with an `Unknown` outcome across r Startup errors distinguish damaged recovery metadata or undecodable signed bytes (`ShieldedRecoveryCorrupted`) from missing/replaced viewing keys (`ShieldedRecoveryKeysRequired`). Failure to recover an output set can also mean damaged ciphertext or output metadata; restore the original keys or a known-good backup before choosing explicit recovery. Invalid SQLite identifiers, nullifiers, and state flags fail startup without deleting their rows. When signed bytes are damaged but the recovery key remains readable, listing retains its account/activity identifiers and leaves the undecodable identity, nonce, and amount absent. +## Private contact request payments after a restart + +Before a private contact request's payment can reach the network, the shielded store saves a record of it in its private payment journal: the sender and recipient identities, the request's account reference, the commitment of the entry it appends and the nullifiers its spend consumes. The file store writes the journal, and the payment's redrive record, in one transaction on a connection at `synchronous=FULL`, so the record survives a crash or a power loss; a record that cannot be saved refuses the payment before anything is broadcast. The journal is kept apart from the spend's notes and redrive record: no note release, redrive removal, undecodable transition, account rebind or purge removes it. A record is removed only when its entry is read back from DashPay's private store and recorded durably, or when the transition provably never left the wallet (refused before anything was sent: no address to send it to, or the SDK's own refusal). An endpoint's answer, a consensus verdict included, proves nothing: no proof or quorum signature backs it, and an endpoint that kept or forwarded the signed transition can answer with any verdict. A record is also discharged once its payment is proven expired: the shielded sync pass finds the spend's anchor, which the record keeps, absent from Platform's recorded anchors fetched before the pass, one of its input notes still unspent after the pass's spent-note reconcile, and its entry absent from DashPay's private store read after that. A payment that never lands therefore frees its pair once its anchor ages out (Platform keeps an anchor for `shielded_anchor_retention_blocks`, 1000 blocks, and prunes every `shielded_anchor_pruning_interval`, 100 blocks). Nothing else, a relay's refusal or a missing redrive record included, discharges a record. + +Until a payment's reservation is restored from the journal, nothing shows that the pair is taken: the request's entry is not in DashPay's store until the payment executes. So a reloaded wallet sends and accepts no contact request, public or private, while such a payment may be unrestored, and returns `PrivateContactPaymentsNotRestored` (FFI code 72) with nothing read, signed or broadcast. The in-kind accept is held the same way. Two things hold it: + +- In a build with the `shielded` feature, `load_from_persistor` holds a wallet that bound a shielded account before (its persisted viewing key shows it). A wallet with no persisted viewing key never bound one, since a bind persists the key before installing anything and refuses when it cannot. +- In every build, an identity whose private scan state marks a private payment that may be unresolved (`PrivateContactRequestsPosition::private_payments_unresolved`) is held. The wallet stores the marker durably before such a payment may reach the network, flushing a backend that does not commit a store inline, and stores it cleared once the identity holds no private request it is paying for or has not read back. A loaded hold is kept even when the backend no longer attests the scan state. + +A private contact request is therefore paid for only over a backend attesting `PRIVATE_CONTACT_REQUEST_SCAN_STATE` together with `ATOMIC_CHANGESETS` (`PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART`), as the scan state is written in changesets that must commit whole: any other refuses the private send and the private accept with `PrivateContactRecoveryUnsupported` (FFI code 73) before anything is sealed or paid, and a marker that cannot be stored or flushed refuses the payment too. A payment that may have reached the network keeps its pair held and its marker set whatever its spend's outcome, until its record is removed as above. A pair held so refuses another private payment (`PrivateContactRequestPending`, FFI code 71) and holds public sends and accepts. + +`PlatformWalletManager::configure_shielded` restores the payments of every held wallet from the journal, with no keys, and a load after it restores those it loads. Binding a wallet's shielded account restores its own; a bind never waits for a store that a scan holds, and defers the restoration to a task instead, the wallet held until it succeeds. A private sync does not restore. Hosts that use shielded support call `configure_shielded` right after the load. A host that opens no shielded store this session, and any build without the `shielded` feature, keeps those wallets' and identities' contact requests refused: the private payment may still execute, and a public request would publish the pair the private one kept off chain. `DashPayView::private_contact_readiness` (FFI `platform_wallet_get_private_contact_readiness`) reads whether the payments are restored, whether the private store was read to its end since load, and whether the marker is set, so a host shows the wallet's state rather than inferring it from refusals. + +A restoration fails closed, every hold staying, when the coordinator is for another network, when a journal record does not read back, when a saved spend's transition does not decode and no journal record accounts for it, when a private contact request payment saved before the journal kept it has no request saved with it, and when an identity is marked as holding an unresolved payment while the wallet's journal is new (the shielded store was deleted or replaced, so it cannot show that payment resolved). The first three clear once the damaged spend lands or its anchor is pruned, which removes its redrive record, or once a known-good store file is restored; the last one only once a known-good store is restored, or by removing the wallet and adding it again when the payment is known to have resolved. + +### Shielded spends whose broadcast may have been delivered + +Every shielded spend that consumes notes (shielded transfers, unshields, withdrawals, identity top-ups and creations from the pool, and private document creations) broadcasts in single dispatches and keeps what each one proved: the SDK's own retry loop returns only its last error, which cannot show that an earlier attempt was admitted with its response lost. Any dispatch that reached an endpoint, whatever it answered (a consensus verdict included, which is unauthenticated), or failed in a way that allows delivery, any transport error whatever its gRPC code, makes the whole broadcast possibly delivered. Such a spend waits for its result, and reports `ShieldedSpendUnconfirmed` (FFI code 18) when that wait fails or answers with a verdict, its notes and redrive record kept so the redrive pass resolves it: a host must not pay again with other notes. Only a spend refused before anything was sent reports `ShieldedBroadcastFailed` and releases its notes at once. The dispatches stay within the SDK's configured retry budget: one dispatch and its retries. A re-broadcast refused by a node does not prove that no other node holds the original, so the redrive pass keeps such a spend's notes reserved until it lands or its anchor is pruned. Shields into the pool (from platform addresses or an identity) keep their own handling. + ## Shielded balance API migration +Custom `ShieldedStore` implementations must also implement the private +contact payment methods, which have no default: `arm_private_contact_payment`, +`private_contact_redrives_for_wallet` and the private payment journal +(`open_private_payment_journal`, `private_payment_journal`, +`journal_private_payments`, `discharge_private_payment`). A store that cannot +keep a payment's records durably returns an error, which refuses the payment; +it must never report success while dropping them. + The local shielded balance API introduces three Rust source compatibility changes: - Custom `ShieldedStore` implementations must implement @@ -146,6 +173,12 @@ The local shielded balance API introduces three Rust source compatibility change index is zero. An absent row is different from a recorded empty scan; do not infer presence merely from `last_synced_index > 0`. +## Shielded store lifetime migration + +`ShieldedStore` now requires `Send + Sync + 'static`: a spend dropped before it hands its reserved notes over releases them on a spawned task when the store is locked, and that task owns the store's `Arc`. This is a Rust source compatibility change, not a consensus one. + +A custom `ShieldedStore` that borrows from its embedding application (a `&'a` connection, configuration or buffer held in the store type) no longer satisfies the trait. Move the borrowed resource into the store, by value or behind an `Arc` (an `Arc>` or `Arc>` for one shared with the application), so the store type has no lifetime parameter. The in-repo [InMemoryShieldedStore](src/wallet/shielded/store.rs) and [FileBackedShieldedStore](src/wallet/shielded/file_store.rs) already own what they use and need no change. + ## Dependencies - `key-wallet`: Core wallet functionality diff --git a/packages/rs-platform-wallet/src/broadcast_outcome.rs b/packages/rs-platform-wallet/src/broadcast_outcome.rs index 82648b66799..bcb54c1983e 100644 --- a/packages/rs-platform-wallet/src/broadcast_outcome.rs +++ b/packages/rs-platform-wallet/src/broadcast_outcome.rs @@ -2,9 +2,13 @@ //! //! Shared by every Platform money-moving path that must keep "definitely //! rejected, safe to retry" apart from "ambiguous, may have executed, do -//! not retry" (shielded spends, masternode credit withdrawals). The shapes -//! are SDK-transport facts, not operation-specific, so they live here -//! rather than behind the `shielded` feature. +//! not retry". Shielded spends that consume notes broadcast through +//! [`broadcast_tracking_delivery`], which keeps every dispatch's outcome so an +//! earlier dispatch that may have been delivered is never hidden by a later +//! refusal; shields into the pool and masternode credit withdrawals classify +//! the SDK's final error with [`broadcast_definitely_failed`]. The shapes are +//! SDK-transport facts, not operation-specific, so they live here rather than +//! behind the `shielded` feature. /// Whether an SDK error carries Platform's own consensus verdict on the /// transition. Two shapes qualify: @@ -56,13 +60,12 @@ pub(crate) fn carries_consensus_rejection(err: &dash_sdk::Error) -> bool { /// stream resets after the request bytes left can surface the same code, /// and the dapi-client's cross-address retry only retains the LAST /// transport error, so an earlier-attempt delivery can hide behind a -/// later attempt's `Unavailable`. Releasing the notes in that residual -/// window is still fund-safe — the authoritative no-reuse guarantee is -/// the on-chain nullifier set, so a re-selected note at worst wastes a -/// ~30 s proof on a nullifier-already-used rejection (see the -/// `finalize_pending` downgrade rationale in `unshield`); never fund -/// loss. The trade is deliberate: UX for the dominant offline case over -/// strict conservatism in a rare race. +/// later attempt's final error, whatever its code. That is why shielded +/// spends do not use this classification but [`broadcast_tracking_delivery`], +/// which reports such a spend unconfirmed: a spend reported failed while it +/// may still execute could be paid again with other notes. The callers left +/// here are shields into the pool, which move funds to the wallet's own +/// notes, and masternode credit withdrawals. /// /// Everything else leaves the outcome unknown and the caller must fall /// through to the result wait instead of failing: `AlreadyExists` proves @@ -105,3 +108,373 @@ pub(crate) fn broadcast_definitely_failed(e: &dash_sdk::Error) -> bool { _ => false, } } + +/// What the dispatches of one shielded spend's broadcast proved about its +/// delivery ([`broadcast_tracking_delivery`]). +#[cfg(feature = "shielded")] +#[derive(Debug)] +pub(crate) enum BroadcastDelivery { + /// A node admitted the transition, or already held it (`AlreadyExists`). + Admitted, + /// No dispatch proved the transition was left out of every mempool: a + /// dispatch reached an endpoint, which may have kept or forwarded it, + /// whatever it answered (a consensus verdict included, which no proof + /// or quorum signature backs), or failed in a way that allows delivery. + /// It may execute, so the caller waits for its result, and an + /// unconfirmed outcome keeps the notes reserved. + MaybeDelivered(dash_sdk::Error), + /// The transition never left the wallet: it was refused before its first + /// dispatch was sent, there being no address to send it to or the SDK + /// refusing it, with no dispatch sent before. Nothing holds its bytes but + /// the wallet, so it can never execute. + NeverDelivered(dash_sdk::Error), +} + +/// How many single dispatches [`broadcast_tracking_delivery`] makes at most +/// on `sdk`: its configured overall budget, one dispatch and its effective +/// retries, so a client configured with no retries dispatches once. +#[cfg(feature = "shielded")] +pub(crate) fn spend_broadcast_dispatches(sdk: &dash_sdk::Sdk) -> usize { + sdk.query_settings() + .request_settings + .finalize() + .retries + .saturating_add(1) +} + +/// Broadcast a shielded spend `state_transition` in single dispatches (no +/// SDK retry inside a dispatch), at most the SDK's configured budget +/// ([`spend_broadcast_dispatches`]), keeping what every dispatch proved: the +/// SDK's own retry loop returns only its last error, which cannot show that +/// an earlier attempt was delivered with its response lost. Centralizing that +/// evidence in the SDK is dashpay/platform#5556; what it means for a payment's +/// notes, journal and contact stays here. +/// +/// Any dispatch that reached an endpoint, or failed in a way that allows it +/// (every transport error, whatever its gRPC code, a timeout, a cause-less +/// broadcast envelope, an unknown error), makes the whole broadcast +/// [`BroadcastDelivery::MaybeDelivered`]. So does a consensus verdict: an +/// endpoint's answer is unauthenticated, and one that kept or forwarded the +/// signed transition can answer with any verdict. Only a refusal before +/// anything was sent is [`BroadcastDelivery::NeverDelivered`]. +#[cfg(feature = "shielded")] +pub(crate) async fn broadcast_tracking_delivery( + sdk: &dash_sdk::Sdk, + state_transition: &dpp::state_transition::StateTransition, +) -> BroadcastDelivery { + use dash_sdk::dapi_client::RequestSettings; + use dash_sdk::platform::transition::broadcast::BroadcastStateTransition; + use dash_sdk::platform::transition::put_settings::PutSettings; + + let single_dispatch = PutSettings { + request_settings: RequestSettings { + retries: Some(0), + ..RequestSettings::default() + }, + ..PutSettings::default() + }; + track_delivery(spend_broadcast_dispatches(sdk), || { + state_transition.broadcast(sdk, Some(single_dispatch)) + }) + .await +} + +/// The classification [`broadcast_tracking_delivery`] makes, over the +/// outcomes `dispatch` returns, one call per single dispatch, at most +/// `dispatches` of them. +#[cfg(feature = "shielded")] +async fn track_delivery(dispatches: usize, mut dispatch: F) -> BroadcastDelivery +where + F: FnMut() -> Fut, + Fut: std::future::Future>, +{ + use dash_sdk::dapi_client::DapiClientError; + + let mut maybe_delivered: Option = None; + for _ in 0..dispatches.max(1) { + let error = match dispatch().await { + Ok(()) | Err(dash_sdk::Error::AlreadyExists(_)) => return BroadcastDelivery::Admitted, + Err(error) => error, + }; + // Refused before anything was sent: no address, or the SDK's own + // refusal of a kind the network does not serve yet. + let never_sent = matches!( + error, + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses) + | dash_sdk::Error::ProtocolVersionNotActive { .. } + ) || matches!( + &error, + dash_sdk::Error::NoAvailableAddressesToRetry(inner) + if matches!( + **inner, + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses) + ) + ); + if never_sent { + return match maybe_delivered { + Some(_) => BroadcastDelivery::MaybeDelivered(error), + None => BroadcastDelivery::NeverDelivered(error), + }; + } + // An endpoint's verdict ends the dispatches, and proves nothing. + if carries_consensus_rejection(&error) { + return BroadcastDelivery::MaybeDelivered(error); + } + maybe_delivered = Some(error); + } + match maybe_delivered { + Some(error) => BroadcastDelivery::MaybeDelivered(error), + // Unreachable: every pass sets it or returns. + None => BroadcastDelivery::MaybeDelivered(dash_sdk::Error::Generic( + "no broadcast dispatch was made".to_string(), + )), + } +} + +#[cfg(all(test, feature = "shielded"))] +mod tests { + use super::*; + use dash_sdk::dapi_client::transport::TransportError; + use dash_sdk::dapi_client::DapiClientError; + use dash_sdk::dapi_grpc::tonic::{Code, Status}; + use std::sync::Mutex; + + /// Another error of the same shape as `error`, which is not `Clone`. + fn copy(error: &dash_sdk::Error) -> dash_sdk::Error { + match error { + dash_sdk::Error::DapiClientError(DapiClientError::Transport(TransportError::Grpc( + status, + ))) => grpc(status.code()), + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses) => { + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses) + } + dash_sdk::Error::NoAvailableAddressesToRetry(inner) => { + dash_sdk::Error::NoAvailableAddressesToRetry(Box::new(copy(inner))) + } + dash_sdk::Error::Protocol(dpp::ProtocolError::ConsensusError(_)) => rejection(), + dash_sdk::Error::ProtocolVersionNotActive { + method, + required, + feature, + network, + pinned, + .. + } => dash_sdk::Error::ProtocolVersionNotActive { + method, + required: *required, + feature: *feature, + network: *network, + pinned: *pinned, + cause: None, + }, + other => dash_sdk::Error::Generic(other.to_string()), + } + } + + fn grpc(code: Code) -> dash_sdk::Error { + dash_sdk::Error::DapiClientError(DapiClientError::Transport(TransportError::Grpc( + Status::new(code, "test"), + ))) + } + + /// A consensus rejection, as DAPI attaches one to a refused broadcast. + fn rejection() -> dash_sdk::Error { + use dpp::consensus::basic::BasicError; + use dpp::consensus::basic::UnsupportedFeatureError; + dash_sdk::Error::Protocol(dpp::ProtocolError::ConsensusError(Box::new( + dpp::consensus::ConsensusError::BasicError(BasicError::UnsupportedFeatureError( + UnsupportedFeatureError::new("test".to_string(), 1), + )), + ))) + } + + /// What [`track_delivery`] makes of dispatches whose outcome `outcome` + /// gives by dispatch number, from 0, and how many dispatches it made. + async fn tracked( + outcome: impl Fn(usize) -> Result<(), dash_sdk::Error>, + ) -> (BroadcastDelivery, usize) { + let dispatched = Mutex::new(0usize); + let delivery = track_delivery(BUDGET, || { + let mut count = dispatched.lock().expect("count"); + let result = outcome(*count); + *count += 1; + std::future::ready(result) + }) + .await; + let dispatched = *dispatched.lock().expect("count"); + (delivery, dispatched) + } + + /// The dispatch budget the tests run with. + const BUDGET: usize = 6; + + /// An endpoint that kept or forwarded the transition can answer the + /// first dispatch with a consensus verdict, as serialized metadata the + /// SDK decodes with no proof behind it: the broadcast stays maybe + /// delivered, so its payment stays unconfirmed with its notes, journal + /// record and contact hold, and the dispatches end there. + #[tokio::test] + async fn should_keep_a_broadcast_an_endpoint_refused_maybe_delivered() { + use dash_sdk::dapi_grpc::tonic::metadata::{MetadataMap, MetadataValue}; + use dpp::serialization::PlatformSerializableWithPlatformVersion; + use dpp::version::PlatformVersion; + + let verdict = || { + let bytes = dpp::consensus::ConsensusError::BasicError( + dpp::consensus::basic::BasicError::UnsupportedFeatureError( + dpp::consensus::basic::UnsupportedFeatureError::new("test".to_string(), 1), + ), + ) + .serialize_to_bytes_with_platform_version(PlatformVersion::latest()) + .expect("serialized"); + let mut metadata = MetadataMap::new(); + metadata.insert_bin( + "dash-serialized-consensus-error-bin", + MetadataValue::from_bytes(&bytes), + ); + dash_sdk::Error::from(DapiClientError::Transport(TransportError::Grpc( + Status::with_metadata(Code::InvalidArgument, "rejected", metadata), + ))) + }; + assert!(carries_consensus_rejection(&verdict()), "{:?}", verdict()); + let (delivery, dispatched) = tracked(|_| Err(verdict())).await; + assert!( + matches!(delivery, BroadcastDelivery::MaybeDelivered(_)), + "{delivery:?}" + ); + assert_eq!(dispatched, 1); + } + + /// The tracker keeps the SDK's configured budget: a client configured + /// with no retries dispatches once, one with three retries four times. + #[tokio::test] + async fn should_keep_the_sdk_s_configured_retry_budget() { + use dash_sdk::dapi_client::RequestSettings; + + let no_retries = dash_sdk::SdkBuilder::new_mock() + .with_settings(RequestSettings { + retries: Some(0), + ..RequestSettings::default() + }) + .build() + .expect("an SDK"); + assert_eq!(spend_broadcast_dispatches(&no_retries), 1); + let three_retries = dash_sdk::SdkBuilder::new_mock() + .with_settings(RequestSettings { + retries: Some(3), + ..RequestSettings::default() + }) + .build() + .expect("an SDK"); + assert_eq!(spend_broadcast_dispatches(&three_retries), 4); + + let dispatched = Mutex::new(0usize); + let delivery = track_delivery(spend_broadcast_dispatches(&no_retries), || { + *dispatched.lock().expect("count") += 1; + std::future::ready(Err(grpc(Code::Unavailable))) + }) + .await; + assert!(matches!(delivery, BroadcastDelivery::MaybeDelivered(_))); + assert_eq!(*dispatched.lock().expect("count"), 1); + } + + /// An attempt admitted with its response lost, then retries refused for + /// any reason, leaves the broadcast maybe delivered: the last error + /// alone cannot show the transition never left. + #[tokio::test] + async fn should_keep_an_earlier_ambiguous_dispatch_over_later_refusals() { + for lost in [Code::Unavailable, Code::DeadlineExceeded] { + for refusal in [ + grpc(Code::ResourceExhausted), + grpc(Code::Unimplemented), + grpc(Code::InvalidArgument), + rejection(), + ] { + let refusal = &refusal; + let (delivery, _) = tracked(|dispatch| match dispatch { + 0 => Err(grpc(lost)), + _ => Err(copy(refusal)), + }) + .await; + assert!( + matches!(delivery, BroadcastDelivery::MaybeDelivered(_)), + "{lost:?} then a refusal: {delivery:?}" + ); + } + } + } + + /// Every transport-level failure may have delivered the transition, + /// whatever its code, so a broadcast that only ever fails so is maybe + /// delivered after the whole budget. + #[tokio::test] + async fn should_treat_every_transport_failure_as_maybe_delivered() { + for code in [ + Code::Unavailable, + Code::ResourceExhausted, + Code::Unimplemented, + Code::InvalidArgument, + Code::Internal, + ] { + let (delivery, dispatched) = tracked(|_| Err(grpc(code))).await; + assert!( + matches!(delivery, BroadcastDelivery::MaybeDelivered(_)), + "{code:?}: {delivery:?}" + ); + assert_eq!(dispatched, BUDGET, "{code:?}"); + } + } + + /// Only a refusal before anything was sent, nowhere to send the + /// transition or the SDK refusing it, with no dispatch sent before, + /// shows it never left. + #[tokio::test] + async fn should_count_a_broadcast_never_delivered_only_without_an_earlier_ambiguity() { + for first in [ + dash_sdk::Error::ProtocolVersionNotActive { + method: "broadcast_state_transition", + required: 15, + feature: None, + network: 14, + pinned: false, + cause: None, + }, + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses), + dash_sdk::Error::NoAvailableAddressesToRetry(Box::new( + dash_sdk::Error::DapiClientError(DapiClientError::NoAvailableAddresses), + )), + ] { + let first = &first; + let (delivery, dispatched) = tracked(|_| Err(copy(first))).await; + assert!( + matches!(delivery, BroadcastDelivery::NeverDelivered(_)), + "{delivery:?}" + ); + assert_eq!(dispatched, 1); + } + } + + /// A dispatch a node admitted, or already holds, is admitted, after any + /// number of failed ones. + #[tokio::test] + async fn should_count_an_admitted_or_known_dispatch_as_admitted() { + let (delivery, _) = tracked(|dispatch| match dispatch { + 0 => Err(grpc(Code::Unavailable)), + _ => Ok(()), + }) + .await; + assert!( + matches!(delivery, BroadcastDelivery::Admitted), + "{delivery:?}" + ); + let (delivery, _) = tracked(|dispatch| match dispatch { + 0 => Err(grpc(Code::DeadlineExceeded)), + _ => Err(dash_sdk::Error::AlreadyExists("in the mempool".to_string())), + }) + .await; + assert!( + matches!(delivery, BroadcastDelivery::Admitted), + "{delivery:?}" + ); + } +} diff --git a/packages/rs-platform-wallet/src/changeset/changeset.rs b/packages/rs-platform-wallet/src/changeset/changeset.rs index 36d3faeef7c..45b333fd055 100644 --- a/packages/rs-platform-wallet/src/changeset/changeset.rs +++ b/packages/rs-platform-wallet/src/changeset/changeset.rs @@ -1298,9 +1298,11 @@ pub struct ReceivedContactRequestKey { /// (a `sent_requests` insert clears a prior `removed_sent`, an un-ignore /// clears a prior ignore, and vice versa), keeping the two sets /// disjoint. This covers `sent_requests` vs `removed_sent`, -/// `incoming_requests` vs `removed_incoming`, and `ignored` vs -/// `unignored`. `established` has no opposing tombstone set and rides -/// plain last-write-wins. +/// `incoming_requests` vs `removed_incoming`, `ignored` vs +/// `unignored`, and `private_shielded_addresses` vs +/// `removed_private_shielded_addresses`. `established` and +/// `private_contact_requests_positions` have no opposing tombstone set +/// and ride plain last-write-wins. #[derive(Debug, Clone, Default, PartialEq)] #[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] pub struct ContactChangeSet { @@ -1330,6 +1332,66 @@ pub struct ContactChangeSet { /// sweep. Kept as a separate set (rather than a shrinking `ignored` /// snapshot) so the changeset's insert-XOR-tombstone discipline holds. pub unignored: BTreeSet<(Identifier, Identifier)>, + /// The position of DashPay's private contact request store each owner + /// identity reads from next, keyed by owner, with the fingerprint of the + /// scanning keys it holds for: every entry before it was read with those + /// keys and what it held for the owner stored. Written only for a backend + /// attesting + /// [`PRIVATE_CONTACT_REQUEST_SCAN_STATE`](crate::changeset::PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE), + /// and only after the stores of what the entries before it held, so a + /// restored position never covers a request that did not reach disk. A + /// rewind (un-ignoring a sender, or the scanning keys changing) writes + /// the lower position. Last write wins. + #[cfg_attr(feature = "serde", serde(default))] + pub private_contact_requests_positions: BTreeMap, + /// The Orchard address each contact's tracked private request to the + /// owner carries, keyed by `(owner, contact)`: 43 bytes, an 11-byte + /// diversifier and a 32-byte transmission key. Written only for a + /// backend attesting + /// [`PRIVATE_CONTACT_REQUEST_SCAN_STATE`](crate::changeset::PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE). + /// Reconciled last-write-wins against + /// [`Self::removed_private_shielded_addresses`] on merge. + #[cfg_attr( + feature = "serde", + serde( + default, + with = "crate::changeset::serde_adapters::private_shielded_addresses" + ) + )] + pub private_shielded_addresses: BTreeMap<(Identifier, Identifier), [u8; 43]>, + /// `(owner, contact)` pairs whose tracked private request carries no + /// shielded address any more: the removal tombstone for + /// [`Self::private_shielded_addresses`]. + #[cfg_attr(feature = "serde", serde(default))] + pub removed_private_shielded_addresses: BTreeSet<(Identifier, Identifier)>, +} + +/// Where an identity reads DashPay's private contact request store from +/// next, and the scanning keys that position holds for. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct PrivateContactRequestsPosition { + /// The store position read from next. + pub position: u64, + /// SHA-256 over the identity's HD index and each `ECDSA_SECP256K1` + /// `ENCRYPTION` or `DECRYPTION` key (id, purpose and data, in key id + /// order) the entries before `position` were scanned with + /// (`ManagedIdentity::private_scan_key_fingerprint`). A loaded position + /// whose fingerprint is not the identity's is read again from 0. + pub key_fingerprint: [u8; 32], + /// Whether a private contact request this identity paid for may still be + /// unresolved: written set before such a payment may reach the network, + /// and cleared once the identity holds no private request it is paying + /// for or has not read back (each read back, definitively rejected, or + /// released before it was broadcast). Loaded set, it holds the identity's + /// public contact requests, sent or accepted, in every build, until a + /// restoration from the shielded store restores the reservations it + /// stands for (`PlatformWallet::restore_private_contact_payments`). A + /// build without the `shielded` feature never restores them, so it keeps + /// the identity held: its private payment may still execute, and a + /// public request would publish the pair the private one kept off chain. + #[cfg_attr(feature = "serde", serde(default))] + pub private_payments_unresolved: bool, } impl Merge for ContactChangeSet { @@ -1370,6 +1432,20 @@ impl Merge for ContactChangeSet { } self.ignored.extend(other.ignored); self.unignored.extend(other.unignored); + + self.private_contact_requests_positions + .extend(other.private_contact_requests_positions); + + for key in other.removed_private_shielded_addresses.iter() { + self.private_shielded_addresses.remove(key); + } + for key in other.private_shielded_addresses.keys() { + self.removed_private_shielded_addresses.remove(key); + } + self.private_shielded_addresses + .extend(other.private_shielded_addresses); + self.removed_private_shielded_addresses + .extend(other.removed_private_shielded_addresses); } fn is_empty(&self) -> bool { @@ -1380,6 +1456,9 @@ impl Merge for ContactChangeSet { && self.established.is_empty() && self.ignored.is_empty() && self.unignored.is_empty() + && self.private_contact_requests_positions.is_empty() + && self.private_shielded_addresses.is_empty() + && self.removed_private_shielded_addresses.is_empty() } } diff --git a/packages/rs-platform-wallet/src/changeset/identity_manager_start_state.rs b/packages/rs-platform-wallet/src/changeset/identity_manager_start_state.rs index a4072b61848..f4f993c3463 100644 --- a/packages/rs-platform-wallet/src/changeset/identity_manager_start_state.rs +++ b/packages/rs-platform-wallet/src/changeset/identity_manager_start_state.rs @@ -37,3 +37,23 @@ pub struct IdentityManagerStartState { /// forces a rescan. See [`IdentityScanStateEntry`]. pub scan_states: BTreeMap, } + +impl IdentityManagerStartState { + /// Drop the private contact request scan state every restored identity + /// carries (its read position and its contacts' shielded addresses), as a + /// load does for a backend that does not attest + /// [`PRIVATE_CONTACT_REQUEST_SCAN_STATE`](crate::changeset::PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE): + /// nothing keeps such a backend's copy current, so the store is read from + /// its start. A restored hold on an unresolved private payment is kept + /// (`ManagedIdentity::forget_private_contact_scan_state`). + pub(crate) fn forget_private_contact_scan_state(&mut self) { + for managed in self + .wallet_identities + .values_mut() + .flat_map(BTreeMap::values_mut) + .chain(self.out_of_wallet_identities.values_mut()) + { + managed.forget_private_contact_scan_state(); + } + } +} diff --git a/packages/rs-platform-wallet/src/changeset/mod.rs b/packages/rs-platform-wallet/src/changeset/mod.rs index f246d3f6056..d61e3eae156 100644 --- a/packages/rs-platform-wallet/src/changeset/mod.rs +++ b/packages/rs-platform-wallet/src/changeset/mod.rs @@ -40,9 +40,9 @@ pub use changeset::{ IdentityKeysChangeSet, InvitationChangeSet, InvitationEntry, InvitationStatus, KeyDerivationBreadcrumb, KeyWithBreadcrumb, PendingContactCrypto, PendingContactCryptoKey, PendingContactCryptoKind, PendingContactCryptoOp, PlatformAddressBalanceEntry, - PlatformAddressChangeSet, PlatformWalletChangeSet, ProviderKeyAccountEntry, - ProviderKeyExtendedPubKey, ProviderPlatformNodePubKey, ReceivedContactRequestKey, - SentContactRequestKey, TokenBalanceChangeSet, WalletMetadataEntry, + PlatformAddressChangeSet, PlatformWalletChangeSet, PrivateContactRequestsPosition, + ProviderKeyAccountEntry, ProviderKeyExtendedPubKey, ProviderPlatformNodePubKey, + ReceivedContactRequestKey, SentContactRequestKey, TokenBalanceChangeSet, WalletMetadataEntry, }; pub use client_start_state::ClientStartState; pub use client_wallet_start_state::ClientWalletStartState; diff --git a/packages/rs-platform-wallet/src/changeset/persistence_capabilities.rs b/packages/rs-platform-wallet/src/changeset/persistence_capabilities.rs index 6c17b550b1e..9a2913f0792 100644 --- a/packages/rs-platform-wallet/src/changeset/persistence_capabilities.rs +++ b/packages/rs-platform-wallet/src/changeset/persistence_capabilities.rs @@ -111,13 +111,31 @@ impl PersistenceCapabilities { /// empty result. Required before a wallet drain can treat absence as not sent. pub const ASSET_LOCK_OPERATION_IDS: Self = Self(1 << 13); + /// The private contact request sync's scan state round-trips: each + /// identity's read position in DashPay's private contact request store + /// (`ContactChangeSet::private_contact_requests_positions`) and each + /// private contact's shielded address + /// (`ContactChangeSet::private_shielded_addresses`, with its removals) + /// are stored, and loading restores them onto each `ManagedIdentity` + /// (`ManagedIdentity::apply_private_contact_requests_position`, + /// `ManagedIdentity::apply_private_shielded_address`), so a relaunch + /// reads the store from where it stopped. Without this bit nothing of + /// it is written, a restored value is dropped, and each launch reads the + /// store from its start. + /// + /// Honoured only together with [`Self::ATOMIC_CHANGESETS`] + /// ([`Self::PRIVATE_CONTACT_SCAN_STATE_RESTART`]): the scan state is written + /// in changesets that must commit whole, so a backend declaring this bit + /// alone is treated as keeping none of it. + pub const PRIVATE_CONTACT_REQUEST_SCAN_STATE: Self = Self(1 << 14); + /// Index of the highest bit declared above. It lives here, beside the /// constants, so adding a bit and bumping this is one edit in one place /// — and `every_declared_bit_has_a_stable_name` walks up to it, so a new /// bit that never reaches `KNOWN` fails a test instead of gating /// behaviour invisibly. The same test asserts nothing above it is named, /// which is what catches a bit added without bumping this. - const HIGHEST_DECLARED_BIT: u32 = 13; + const HIGHEST_DECLARED_BIT: u32 = 14; /// Capabilities required before exporting and funding an invitation voucher. pub const INVITATION_CREATION: Self = Self( @@ -131,6 +149,16 @@ impl PersistenceCapabilities { pub const SHIELDED_FVK_RESTART: Self = Self(Self::ATOMIC_CHANGESETS.0 | Self::SHIELDED_VIEWING_KEYS.0); + /// Capabilities required for the private contact request scan state to + /// round-trip, and so for a private contact request to be paid for: the + /// state itself, and atomic changesets. An un-ignore stores the sender's + /// removal and the read-position rewind it brings in one changeset; a + /// backend committing the removal without the rewind would, after a + /// restart, read on past the requests passed over while the sender was + /// ignored, and nothing would read them again. + pub const PRIVATE_CONTACT_SCAN_STATE_RESTART: Self = + Self(Self::ATOMIC_CHANGESETS.0 | Self::PRIVATE_CONTACT_REQUEST_SCAN_STATE.0); + /// Capabilities required to durably reconcile an asset-lock status and /// restore that exact row after process restart. pub const ASSET_LOCK_RECONCILIATION: Self = @@ -221,6 +249,10 @@ impl PersistenceCapabilities { PersistenceCapabilities::ASSET_LOCK_OPERATION_IDS, "asset_lock_operation_ids", ), + ( + PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE, + "private_contact_request_scan_state", + ), ]; KNOWN @@ -238,7 +270,7 @@ impl PersistenceCapabilities { /// failure the test exists to catch. Written as a module-level `const _` so /// it is evaluated in every build, test or not. const _: () = assert!( - PersistenceCapabilities::ASSET_LOCK_OPERATION_IDS.bits() + PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE.bits() == 1u64 << PersistenceCapabilities::HIGHEST_DECLARED_BIT, "HIGHEST_DECLARED_BIT must name the highest declared capability bit" ); @@ -270,6 +302,10 @@ mod tests { PersistenceCapabilities::ASSET_LOCK_OPERATION_IDS.bits(), 0x2000 ); + assert_eq!( + PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE.bits(), + 0x4000 + ); assert_eq!(PersistenceCapabilities::WALLET_DRAIN.bits(), 0x2285); assert_eq!( PersistenceCapabilities::ASSET_LOCK_RECONCILIATION.bits(), diff --git a/packages/rs-platform-wallet/src/changeset/serde_adapters.rs b/packages/rs-platform-wallet/src/changeset/serde_adapters.rs index a4dddbd2092..379e0a99260 100644 --- a/packages/rs-platform-wallet/src/changeset/serde_adapters.rs +++ b/packages/rs-platform-wallet/src/changeset/serde_adapters.rs @@ -11,6 +11,44 @@ use dpp::prelude::AddressNonce; use key_wallet::wallet::managed_wallet_info::asset_lock_builder::AssetLockFundingType; use serde::{Deserialize, Deserializer, Serialize, Serializer}; +/// Adapter for the private shielded addresses of a contact changeset: +/// serde derives no 43-byte array, so each `(owner, contact)` address rides +/// as an `(owner, contact, bytes)` row, its length checked on the way back. +pub mod private_shielded_addresses { + use super::*; + use dpp::prelude::Identifier; + use serde::de::Error as _; + use std::collections::BTreeMap; + + type Addresses = BTreeMap<(Identifier, Identifier), [u8; 43]>; + + pub fn serialize( + addresses: &Addresses, + serializer: S, + ) -> Result { + serializer.collect_seq( + addresses + .iter() + .map(|((owner, contact), address)| (owner, contact, address.as_slice())), + ) + } + + pub fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result { + Vec::<(Identifier, Identifier, Vec)>::deserialize(deserializer)? + .into_iter() + .map(|(owner, contact, address)| { + let address: [u8; 43] = address.as_slice().try_into().map_err(|_| { + D::Error::custom(format!( + "a private shielded address is 43 bytes, not {}", + address.len() + )) + })?; + Ok(((owner, contact), address)) + }) + .collect() + } +} + /// Adapter for `AssetLockFundingType` (upstream has no serde derive). /// /// Encodes each variant as a stable u8 tag — same tag space the diff --git a/packages/rs-platform-wallet/src/error.rs b/packages/rs-platform-wallet/src/error.rs index 322530e770b..11a8c3afd8e 100644 --- a/packages/rs-platform-wallet/src/error.rs +++ b/packages/rs-platform-wallet/src/error.rs @@ -655,6 +655,66 @@ pub enum PlatformWalletError { #[error("SDK error: {0}")] Sdk(#[from] dash_sdk::Error), + /// The contact requests with this identity came or went privately + /// (protocol version 15): a `contactRequest` document to it, in reply or + /// as a rotation, would publish the relationship, so it is sent privately. + #[error( + "the contact requests with {0} came or went privately; reply to or rotate with it privately" + )] + PrivateContactRequestNeedsPrivateReply(Identifier), + + /// A private contact request to this identity is being paid for, or was + /// paid for and is not yet read back from DashPay's store: no other one is + /// sent, privately or publicly, until the private sync reads it back or + /// its payment definitely fails, as the store keeps every entry appended. + /// Also returned, with nothing sealed, paid or broadcast, by the first + /// send or accept from an identity since load, public or private, while + /// DashPay's store cannot yet be read to its end for it: a request paid + /// for in an earlier process may be there unread. Syncing private contact + /// requests and trying again goes on from where that read stopped. + #[error( + "a private contact request to {0} is in flight or not yet read back from the store; sync \ + private contact requests to settle it" + )] + PrivateContactRequestPending(Identifier), + + /// This wallet bound a shielded account before this process started, or + /// the identity was loaded with its private scan state marking a private + /// payment that may be unresolved + /// (`PrivateContactRequestsPosition::private_payments_unresolved`), so the + /// shielded store may hold a private contact request payment from an + /// earlier process that is still unconfirmed, and its reservations are not + /// restored yet: no public contact request is sent or accepted, nothing + /// signed or broadcast, until they are. Opening the shielded store + /// (`PlatformWalletManager::configure_shielded`) restores them, with no + /// keys; so does binding the wallet's shielded account. A private sync + /// does not, and a build without the `shielded` feature never does. + #[error( + "this wallet's unconfirmed private contact request payments from an earlier process are \ + not restored yet; configure the shielded store or bind the wallet's shielded account, \ + then try again" + )] + PrivateContactPaymentsNotRestored, + + /// A private contact request is paid for only with a persister that keeps + /// its recovery marker durably + /// (`PrivateContactRequestsPosition::private_payments_unresolved`), which + /// it declares with + /// [`PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART`](crate::changeset::PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART), + /// the scan state and atomic changesets: + /// without it, a process reopening the wallet, in a build without the + /// `shielded` feature or before the shielded store is configured, could + /// not tell that the payment may still execute, and would publish the + /// pair publicly. Refused before anything is sealed or paid. A host + /// persisting the private contact request scan state in changesets it + /// commits whole declares both. + #[error( + "private contact requests are paid for only with a persister that keeps the private \ + contact request scan state; declare PRIVATE_CONTACT_REQUEST_SCAN_STATE and \ + ATOMIC_CHANGESETS" + )] + PrivateContactRecoveryUnsupported, + /// No DPNS `domain` document exists for the requested name (exact /// normalized-label lookup came back empty). Distinct from /// [`Self::InvalidParameter`]: the input was well-formed, the name @@ -961,6 +1021,16 @@ pub enum PlatformWalletError { #[error("Shielded broadcast failed: {0}")] ShieldedBroadcastFailed(String), + /// A shielded spend's broadcast failed in a way that does not prove the + /// transition was never delivered: an `Unavailable` status, which can + /// follow delivery. Its notes are released, as for + /// [`Self::ShieldedBroadcastFailed`], since the on-chain nullifier set + /// keeps funds safe, but the transition may still execute: a private + /// contact request it pays for keeps its pair held, as for one paid for + /// and not read back. + #[error("Shielded broadcast failed, but may have been delivered: {0}")] + ShieldedBroadcastMaybeDelivered(String), + /// The shielded identity-create transition was **broadcast and accepted by the relay**, but the /// SDK could not confirm its execution result (the result-proof fetch/verify failed — e.g. a /// transient DAPI/proof error, not a platform rejection). The identity with `identity_id` may diff --git a/packages/rs-platform-wallet/src/lib.rs b/packages/rs-platform-wallet/src/lib.rs index 0f4f53b2e4f..70e7cb442f0 100644 --- a/packages/rs-platform-wallet/src/lib.rs +++ b/packages/rs-platform-wallet/src/lib.rs @@ -77,7 +77,8 @@ pub use wallet::core_address_key::{CoreAddressKeyError, CoreAddressPrivateKey}; pub use wallet::identity::network::{ derive_identity_auth_keypair, fetch_identity_public_keys, AutoAcceptProofSource, ContactCryptoProvider, ContactInfoOpened, ContactInfoPublishOutcome, ContactInfoSealed, - FetchedIdentityPublicKeys, PreparedDashPayPayment, SeedBindingVerification, IDENTITY_GAP_LIMIT, + FetchedIdentityPublicKeys, PaymentStarted, PreparedDashPayPayment, PrivateContactReadiness, + PrivateContactSyncReport, PrivateDocumentKeyPaths, SeedBindingVerification, IDENTITY_GAP_LIMIT, MASTER_KEY_INDEX, }; pub use wallet::identity::{ @@ -85,8 +86,8 @@ pub use wallet::identity::{ derive_contact_payment_addresses, derive_contact_xpub, pubkey_binds_expected_key_data, unmask_account_reference, BlockTime, ContactProfileEntry, ContactRequest, ContactXpubData, DashPayProfile, DashPayState, DpnsFetch, DpnsNameInfo, EstablishedContact, IdentityLocation, - IdentityManager, IdentityStatus, KeyStorage, ManagedIdentity, PrivateKeyData, ProfileUpdate, - RegistrationIndex, DEFAULT_CONTACT_GAP_LIMIT, + IdentityManager, IdentityStatus, KeyStorage, ManagedIdentity, PrivateContact, PrivateKeyData, + ProfileUpdate, RegistrationIndex, DEFAULT_CONTACT_GAP_LIMIT, }; pub use wallet::masternode_withdrawal::{ MasternodeWithdrawalKey, MasternodeWithdrawalKeys, MasternodeWithdrawalRequest, @@ -97,7 +98,7 @@ pub use wallet::mnemonic::{ }; pub use wallet::platform_wallet::PlatformWalletInfo; #[cfg(feature = "shielded")] -pub use wallet::platform_wallet::ShieldedShieldPreflight; +pub use wallet::platform_wallet::{PrivateReplyPayment, ShieldedShieldPreflight}; pub use wallet::provider_key_at_index::{ProviderDerivedKey, ProviderKeyKind}; #[cfg(feature = "shielded")] pub use wallet::shielded::operations::shield_fee_reserve_credits; diff --git a/packages/rs-platform-wallet/src/manager/load.rs b/packages/rs-platform-wallet/src/manager/load.rs index 1b612536087..a58b86ce8dd 100644 --- a/packages/rs-platform-wallet/src/manager/load.rs +++ b/packages/rs-platform-wallet/src/manager/load.rs @@ -1,9 +1,13 @@ //! Hydrate a [`PlatformWalletManager`] from its persister. use std::collections::BTreeMap; +#[cfg(feature = "shielded")] +use std::collections::BTreeSet; use std::sync::Arc; -use crate::changeset::{ClientStartState, ClientWalletStartState, PlatformWalletPersistence}; +use crate::changeset::{ + ClientStartState, ClientWalletStartState, PersistenceCapabilities, PlatformWalletPersistence, +}; use crate::error::PlatformWalletError; use crate::wallet::core::WalletGeneration; use crate::wallet::identity::IdentityManager; @@ -86,11 +90,21 @@ impl PlatformWalletManager

{ let ClientStartState { mut platform_addresses, wallets, - // Shielded restore happens lazily on `bind_shielded`, - // not here — drop the snapshot at this entry point. #[cfg(feature = "shielded")] - shielded: _, + shielded, } = start_state; + // Shielded restore happens lazily on `bind_shielded`, not here: only + // which wallets bound a shielded account is kept from the snapshot. + // A bind persists the account's viewing key before anything is + // installed, and refuses when it cannot, so a wallet with no persisted + // viewing key never bound one, and its shielded store holds no private + // contact request payment of its. A Clear deletes both together. + #[cfg(feature = "shielded")] + let bound_shielded: BTreeSet = shielded + .viewing_keys + .keys() + .map(|id| id.wallet_id) + .collect(); // Tracked (wallet-independent) masternodes ride the same startup // hydration; a failure logs and starts empty rather than failing @@ -143,7 +157,7 @@ impl PlatformWalletManager

{ let ClientWalletStartState { mut wallet, mut wallet_info, - identity_manager, + mut identity_manager, unused_asset_locks, unconfirmed_outgoing_txs, dashpay_backfill, @@ -244,7 +258,20 @@ impl PlatformWalletManager

{ observed_input_conflicts: Default::default(), core_wallet: wallet_info, generation: Arc::clone(&generation), - identity_manager: IdentityManager::from(identity_manager), + identity_manager: { + // A backend that does not attest the private contact + // request scan state, with the atomic changesets it needs, + // keeps no current copy of it: the store is read from its + // start. + if !self + .persister + .persistence_capabilities() + .contains(PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART) + { + identity_manager.forget_private_contact_scan_state(); + } + IdentityManager::from(identity_manager) + }, tracked_asset_locks, dpns_name_states: std::collections::BTreeMap::new(), dashpay_backfill, @@ -538,6 +565,17 @@ impl PlatformWalletManager

{ platform_wallet.platform().initialize().await; } + // Before the wallet is published: a public contact request it + // sends or accepts waits until the reservations of its unconfirmed + // private payments from an earlier process are restored from the + // shielded store, which only a configured store can show. + #[cfg(feature = "shielded")] + if bound_shielded.contains(&wallet_id) { + platform_wallet + .persister() + .hold_until_private_payments_restored(); + } + let platform_wallet = Arc::new(platform_wallet); self.wallets.rcu(|wallets| { let mut wallets = std::collections::BTreeMap::clone(wallets); @@ -656,7 +694,16 @@ impl PlatformWalletManager

{ return Err(err); } - // Past the rollback point: every registration here is one this load + // Past the rollback point. A store configured before this load + // restores the held wallets' private payments now, with no keys; + // otherwise `configure_shielded` does. + #[cfg(feature = "shielded")] + if let Some(coordinator) = self.shielded_coordinator().await { + self.restore_held_private_contact_payments(&coordinator) + .await; + } + + // Every registration here is one this load // actually committed, so the transactions now have a wallet to belong // to for as long as it stays registered. // @@ -1893,3 +1940,234 @@ mod tests { ); } } + +#[cfg(all(test, feature = "shielded"))] +mod private_payment_hold_tests { + use std::collections::BTreeMap; + use std::sync::Arc; + + use key_wallet::test_utils::TestWalletContext; + use key_wallet::wallet::ManagedWalletInfo; + use key_wallet::Wallet; + + use crate::changeset::{ + ClientStartState, ClientWalletStartState, IdentityManagerStartState, PersistenceError, + PlatformWalletChangeSet, PlatformWalletPersistence, ShieldedSyncStartState, + }; + use crate::events::PlatformEventHandler; + use crate::test_support::NoopTestEventHandler; + use crate::wallet::identity::ManagedIdentity; + use crate::wallet::platform_wallet::WalletId; + use crate::wallet::shielded::store::PendingRedrive; + use crate::wallet::shielded::{ + FileBackedShieldedStore, PrivateContactPayment, ShieldedStore, SubwalletId, + }; + use crate::PlatformWalletManager; + use dpp::identity::v0::IdentityV0; + use dpp::prelude::Identifier; + + /// The identity the bound wallet manages, which paid for a private + /// contact request to [`RECIPIENT`] in an earlier process. + const SENDER: [u8; 32] = [0xB1; 32]; + /// The identity that private request is to. + const RECIPIENT: [u8; 32] = [0xA1; 32]; + + /// Two wallets, of which only `bound` has a persisted shielded viewing + /// key: it bound a shielded account in an earlier process, and manages + /// [`SENDER`]. + struct ShieldedTracePersister { + wallets: Vec<(Wallet, ManagedWalletInfo)>, + bound: WalletId, + } + + impl PlatformWalletPersistence for ShieldedTracePersister { + fn store( + &self, + _wallet_id: WalletId, + _changeset: PlatformWalletChangeSet, + ) -> Result<(), PersistenceError> { + Ok(()) + } + + fn flush(&self, _wallet_id: WalletId) -> Result<(), PersistenceError> { + Ok(()) + } + + fn load(&self) -> Result { + let wallets = self + .wallets + .iter() + .map(|(wallet, managed)| { + let wallet_id = wallet.compute_wallet_id(); + let mut identity_manager = IdentityManagerStartState::default(); + if wallet_id == self.bound { + let sender = ManagedIdentity::new( + dpp::identity::Identity::V0(IdentityV0 { + id: Identifier::from(SENDER), + public_keys: BTreeMap::new(), + balance: 0, + revision: 1, + }), + 0, + ); + identity_manager + .wallet_identities + .insert(wallet_id, BTreeMap::from([(0, sender)])); + } + ( + wallet_id, + ClientWalletStartState { + wallet: wallet.clone(), + wallet_info: managed.clone(), + identity_manager, + unused_asset_locks: BTreeMap::new(), + unconfirmed_outgoing_txs: Vec::new(), + dashpay_backfill: Default::default(), + }, + ) + }) + .collect(); + let mut shielded = ShieldedSyncStartState::default(); + shielded + .viewing_keys + .insert(SubwalletId::new(self.bound, 0), vec![0; 96]); + Ok(ClientStartState { + wallets, + shielded, + ..Default::default() + }) + } + } + + /// A manager over two wallets, only the first of which bound a shielded + /// account before; with their ids. + fn manager() -> ( + Arc>, + WalletId, + WalletId, + ) { + let bound = TestWalletContext::new_random(); + let unbound = TestWalletContext::new_random(); + let bound_id = bound.wallet.compute_wallet_id(); + let unbound_id = unbound.wallet.compute_wallet_id(); + let persister = ShieldedTracePersister { + wallets: vec![ + (bound.wallet, bound.managed_wallet), + (unbound.wallet, unbound.managed_wallet), + ], + bound: bound_id, + }; + let sdk = Arc::new(dash_sdk::SdkBuilder::new_mock().build().expect("mock sdk")); + let event_handler: Arc = Arc::new(NoopTestEventHandler); + let manager = Arc::new(PlatformWalletManager::new( + sdk, + Arc::new(persister), + event_handler, + )); + (manager, bound_id, unbound_id) + } + + /// Whether `wallet_id`'s public contact requests wait for its private + /// payments to be restored. + async fn held( + manager: &PlatformWalletManager, + wallet_id: WalletId, + ) -> bool { + manager + .get_wallet(&wallet_id) + .await + .expect("loaded") + .persister() + .private_payments_unrestored() + } + + /// Save, in the shielded store at `path`, the bound wallet's payment for a + /// private contact request from [`SENDER`] to [`RECIPIENT`] that may + /// still execute, as an earlier process left it: its redrive record and + /// its journal record. + fn save_payment(path: &std::path::Path, bound: WalletId) { + let mut store = FileBackedShieldedStore::open_path(path, 100).expect("the store"); + store + .arm_private_contact_payment( + SubwalletId::new(bound, 0), + PendingRedrive { + activity_id: [0x21; 32], + anchor: [0x22; 32], + nullifiers: vec![[0x23; 32]], + st_bytes: vec![0xAB; 96], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + PrivateContactPayment { + sender: SENDER, + recipient: RECIPIENT, + account_reference: 0, + }, + &[[0xC1; 32]], + 0, + ) + .expect("the payment saved"); + } + + /// Whether `wallet_id`'s [`SENDER`] holds its pair with [`RECIPIENT`], + /// which refuses every request to it, public or private. + async fn pair_held( + manager: &PlatformWalletManager, + wallet_id: WalletId, + ) -> bool { + let wallet = manager.get_wallet(&wallet_id).await.expect("loaded"); + let wm = wallet.wallet_manager().read().await; + wm.get_wallet_info(&wallet_id) + .and_then(|info| { + info.identity_manager + .managed_identity(&Identifier::from(SENDER)) + }) + .is_some_and(|sender| { + sender + .dashpay() + .private_reply_pending(&Identifier::from(RECIPIENT)) + }) + } + + /// A loaded wallet that bound a shielded account before holds its public + /// contact requests until the shielded store is configured, which + /// restores its private payment's pair with no keys and releases the + /// wallet; one that never bound a shielded account is never held. + #[tokio::test] + async fn should_hold_a_loaded_wallet_that_bound_a_shielded_account_until_the_store_opens() { + let (manager, bound, unbound) = manager(); + let dir = tempfile::tempdir().expect("a directory"); + let path = dir.path().join("tree.sqlite"); + save_payment(&path, bound); + manager.load_from_persistor().await.expect("loaded"); + assert!( + held(&manager, bound).await, + "may have an unrestored payment" + ); + assert!(!pair_held(&manager, bound).await, "not restored yet"); + assert!( + !held(&manager, unbound).await, + "never bound a shielded account" + ); + + manager.configure_shielded(&path).await.expect("configured"); + assert!(!held(&manager, bound).await, "restored from the store"); + assert!(pair_held(&manager, bound).await, "the pair is held"); + } + + /// A wallet loaded once the shielded store is configured is restored by + /// the load itself, its private payment's pair held. + #[tokio::test] + async fn should_restore_a_wallet_loaded_once_the_store_is_configured() { + let (manager, bound, _) = manager(); + let dir = tempfile::tempdir().expect("a directory"); + let path = dir.path().join("tree.sqlite"); + save_payment(&path, bound); + manager.configure_shielded(&path).await.expect("configured"); + manager.load_from_persistor().await.expect("loaded"); + assert!(!held(&manager, bound).await, "restored by the load"); + assert!(pair_held(&manager, bound).await, "the pair is held"); + } +} diff --git a/packages/rs-platform-wallet/src/manager/mod.rs b/packages/rs-platform-wallet/src/manager/mod.rs index 63de4e4d49b..49a14217c61 100644 --- a/packages/rs-platform-wallet/src/manager/mod.rs +++ b/packages/rs-platform-wallet/src/manager/mod.rs @@ -47,6 +47,8 @@ use crate::wallet::asset_lock::LockNotifyHandler; use crate::wallet::core::{BalanceUpdateHandler, SpendObservationHandler}; use crate::wallet::identity::network::DashPayPaymentHandler; use crate::wallet::platform_wallet::{PlatformWalletInfo, WalletId}; +#[cfg(feature = "shielded")] +use crate::wallet::shielded::NetworkShieldedCoordinator; use crate::wallet::PlatformWallet; /// Registry key identifying each background worker the manager joins at @@ -900,6 +902,14 @@ impl PlatformWalletManager

{ /// per-wallet bind looks up the coordinator from the manager /// and errors out if it hasn't been configured. /// + /// Once the store is open, restores the reservations of the unconfirmed + /// private contact request payments of every wallet the manager holds + /// whose public contact requests a load held for them, with no keys + /// ([`Self::restore_held_private_contact_payments`]). Call it right after + /// [`Self::load_from_persistor`] for a wallet that bound a shielded + /// account before: until then, its public contact requests are refused + /// with [`PlatformWalletError::PrivateContactPaymentsNotRestored`]. + /// /// Subsequent calls with the **same** `db_path` are a no-op /// (configuration is idempotent at the path level). A second /// call with a **different** path returns @@ -918,6 +928,9 @@ impl PlatformWalletManager

{ let mut slot = self.shielded_coordinator.write().await; if let Some(existing) = slot.as_ref() { if existing.db_path() == db_path.as_path() { + let existing = Arc::clone(existing); + drop(slot); + self.restore_held_private_contact_payments(&existing).await; return Ok(()); } return Err(crate::error::PlatformWalletError::ShieldedStoreError( @@ -969,10 +982,43 @@ impl PlatformWalletManager

{ .on_shielded_tree_progress(leaves_committed, total_target); }, ))); - *slot = Some(coordinator); + *slot = Some(Arc::clone(&coordinator)); + drop(slot); + self.restore_held_private_contact_payments(&coordinator) + .await; Ok(()) } + /// Restore from `coordinator`'s store, with no keys, the reservations of + /// the unconfirmed private contact request payments of every wallet this + /// manager holds whose public contact requests wait for them + /// ([`PlatformWallet::restore_private_contact_payments`]): a wallet loaded + /// after binding a shielded account in an earlier process, or with an + /// identity marking a private payment that may be unresolved + /// ([`PlatformWallet::private_payments_held`]). A wallet whose + /// restoration fails stays held, until a bind or a later call restores it. + #[cfg(feature = "shielded")] + pub(crate) async fn restore_held_private_contact_payments( + &self, + coordinator: &Arc, + ) { + let wallets: Vec> = self.wallets.load().values().cloned().collect(); + for wallet in wallets { + if !wallet.private_payments_held().await { + continue; + } + if let Err(error) = wallet.restore_private_contact_payments(coordinator).await { + tracing::warn!( + wallet_id = %hex::encode(wallet.wallet_id()), + %error, + "unconfirmed private contact request payments not restored; the wallet's \ + public contact requests stay held until a bind or a later configure \ + restores them" + ); + } + } + } + /// Snapshot of the currently-installed shielded coordinator, /// or `None` if `configure_shielded` hasn't run yet on this /// manager. Cloned `Arc` so callers can hold the coordinator diff --git a/packages/rs-platform-wallet/src/wallet/apply.rs b/packages/rs-platform-wallet/src/wallet/apply.rs index 25b8022e280..96d89dd0908 100644 --- a/packages/rs-platform-wallet/src/wallet/apply.rs +++ b/packages/rs-platform-wallet/src/wallet/apply.rs @@ -234,6 +234,9 @@ impl PlatformWalletInfo { established, ignored, unignored, + private_contact_requests_positions, + private_shielded_addresses, + removed_private_shielded_addresses, } = contact_cs; for (key, entry) in sent_requests { @@ -303,6 +306,23 @@ impl PlatformWalletInfo { managed.apply_unignored_sender(&sender_id); } } + // The private contact request scan state: each owner's read + // position, and its contacts' shielded addresses, removals last. + for (owner_id, position) in private_contact_requests_positions { + if let Some(managed) = self.identity_manager.managed_identity_mut(&owner_id) { + managed.apply_private_contact_requests_position(position); + } + } + for ((owner_id, contact_id), address) in private_shielded_addresses { + if let Some(managed) = self.identity_manager.managed_identity_mut(&owner_id) { + managed.apply_private_shielded_address(contact_id, address); + } + } + for (owner_id, contact_id) in removed_private_shielded_addresses { + if let Some(managed) = self.identity_manager.managed_identity_mut(&owner_id) { + managed.apply_removed_private_shielded_address(&contact_id); + } + } } // 3b. DashPay profile/payment overlays. Applied AFTER identities diff --git a/packages/rs-platform-wallet/src/wallet/identity/mod.rs b/packages/rs-platform-wallet/src/wallet/identity/mod.rs index 01079577e1b..37eb12adeff 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/mod.rs @@ -31,7 +31,8 @@ pub use crypto::{ }; pub use network::{DashPayView, IdentityWallet}; pub use state::{ - BlockTime, DashPayState, IdentityLocation, IdentityManager, ManagedIdentity, RegistrationIndex, + BlockTime, DashPayState, IdentityLocation, IdentityManager, ManagedIdentity, PrivateContact, + RegistrationIndex, }; pub use types::dashpay::profile::{calculate_avatar_hash, calculate_dhash_fingerprint}; pub use types::{ diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs b/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs index 9d7c6664101..e3fada24bff 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs @@ -22,6 +22,17 @@ use crate::wallet::identity::types::dashpay::contact_request::ContactRequest; use crate::wallet::identity::types::dashpay::established_contact::EstablishedContact; use crate::wallet::identity::ManagedIdentity; use crate::wallet::persister::WalletPersister; +use crate::wallet::platform_wallet::{PlatformWalletInfo, WalletId}; +use key_wallet_manager::WalletManager; +use std::sync::Arc; +use tokio::sync::RwLock; + +use super::private_contact_requests::apply_detached; +use dash_sdk::platform::dashpay::PrivateContactRequest; +use dash_sdk::platform::private_document_entries::PrivateDocumentEntries; +use dash_sdk::platform::private_document_scan::PrivateDocumentScan; +use dpp::identity::KeyID; +use dpp::platform_value::BinaryData; use std::collections::BTreeMap; // --------------------------------------------------------------------------- @@ -144,6 +155,87 @@ pub trait ContactCryptoProvider { enc_to_user_id: &[u8; 32], private_data_blob: &[u8], ) -> Result; + + /// Sorts `page`, read from `start_position` of the store of the private document type + /// `document_type_name` of `data_contract_id`, with the keys at `keys`: the entries the + /// wallet's identities received, those they sent, and those refused + /// ([`dash_sdk::platform::private_document_scan::PrivateDocumentScanKeys::scan_page`]). + /// The keys stay in the signer. + /// + /// Defaulted to an "unsupported" error so adding this method is not a source-breaking + /// change: a provider that never reads private contact requests needs no override. + async fn scan_private_document_page( + &self, + _keys: &PrivateDocumentKeyPaths, + _data_contract_id: Identifier, + _document_type_name: &str, + _start_position: u64, + _page: &PrivateDocumentEntries, + ) -> Result { + Err(private_documents_unsupported()) + } + + /// Sorts each of `pages`, each read from its start position of the same store, as + /// [`Self::scan_private_document_page`] sorts one: one scan per page, in their order. A + /// provider derives the keys at `keys` once for all of them, rather than once a page. + /// + /// Defaulted to one [`Self::scan_private_document_page`] call per page, so a provider that + /// only implements that one keeps working. + async fn scan_private_document_pages( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + pages: &[(u64, &PrivateDocumentEntries)], + ) -> Result, PlatformWalletError> { + let mut scans = Vec::with_capacity(pages.len()); + for (start_position, page) in pages { + scans.push( + self.scan_private_document_page( + keys, + data_contract_id, + document_type_name, + *start_position, + page, + ) + .await?, + ); + } + Ok(scans) + } + + /// Seals `request` from `sender`, whose encryption key is the one at + /// `sender_encryption_key_path`, to the key `recipient_key_id` of `recipient`, with + /// `signature`, the signature of the request by the sender's key `signing_key_id` + /// ([`dash_sdk::platform::dashpay::seal_private_contact_request`]). The encryption key's + /// secret key, which the outgoing viewing key is derived from, stays in the signer; the + /// signature comes from the identity signer, as a transition's does. `esk_seed` is 32 + /// fresh random bytes, never reused. + /// + /// Defaulted to an "unsupported" error, as [`Self::scan_private_document_page`]. + #[allow(clippy::too_many_arguments)] + async fn seal_private_contact_request( + &self, + _sender_encryption_key_path: &key_wallet::bip32::DerivationPath, + _dashpay_contract: &dpp::prelude::DataContract, + _sender: &Identity, + _sender_encryption_key_id: KeyID, + _signing_key_id: KeyID, + _signature: &[u8; 65], + _recipient: &Identity, + _recipient_key_id: KeyID, + _request: &PrivateContactRequest, + _esk_seed: &[u8; 32], + _platform_version: &dpp::version::PlatformVersion, + ) -> Result { + Err(private_documents_unsupported()) + } +} + +fn private_documents_unsupported() -> PlatformWalletError { + PlatformWalletError::InvalidIdentityData( + "private contact requests are not supported by this crypto provider".to_string(), + ) } /// Result of [`ContactCryptoProvider::contact_info_seal`] — the two DIP-15 @@ -175,6 +267,9 @@ pub struct ContactInfoOpened { #[cfg(test)] pub(crate) struct SeedCryptoProvider { wallet: key_wallet::wallet::Wallet, + /// How many times the private document scan keys were derived: once per scan call, a + /// page's or a batch's. + pub(crate) scan_key_derivations: std::sync::atomic::AtomicUsize, } #[cfg(test)] @@ -185,6 +280,7 @@ impl SeedCryptoProvider { Self { wallet: Wallet::from_seed_bytes(seed, network, WalletAccountCreationOptions::None) .expect("test seed wallet"), + scan_key_derivations: std::sync::atomic::AtomicUsize::new(0), } } } @@ -312,6 +408,96 @@ impl ContactCryptoProvider for SeedCryptoProvider { private_data, }) } + + async fn scan_private_document_page( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + start_position: u64, + page: &PrivateDocumentEntries, + ) -> Result { + self.scan_private_document_pages( + keys, + data_contract_id, + document_type_name, + &[(start_position, page)], + ) + .await? + .pop() + .ok_or_else(|| PlatformWalletError::InvalidIdentityData("no scan".to_string())) + } + + async fn scan_private_document_pages( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + pages: &[(u64, &PrivateDocumentEntries)], + ) -> Result, PlatformWalletError> { + // Once for the batch, as a signer-backed provider resolves its seed once. + self.scan_key_derivations + .fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let keys = keys.scan_keys(|path| self.secret_key_at(path))?; + let target = platform_encryption::PrivateDocumentTarget { + contract_id: data_contract_id.to_buffer(), + document_type_name, + }; + Ok(pages + .iter() + .map(|(start_position, page)| keys.scan_page(&target, *start_position, page)) + .collect()) + } + + async fn seal_private_contact_request( + &self, + sender_encryption_key_path: &key_wallet::bip32::DerivationPath, + dashpay_contract: &dpp::prelude::DataContract, + sender: &Identity, + sender_encryption_key_id: KeyID, + signing_key_id: KeyID, + signature: &[u8; 65], + recipient: &Identity, + recipient_key_id: KeyID, + request: &PrivateContactRequest, + esk_seed: &[u8; 32], + platform_version: &dpp::version::PlatformVersion, + ) -> Result { + let mut encryption_secret_key = self.secret_key_at(sender_encryption_key_path)?; + let sealed = dash_sdk::platform::dashpay::seal_private_contact_request( + dashpay_contract, + &dash_sdk::platform::dashpay::PrivateContactRequestSender { + identity: sender, + encryption_key_id: sender_encryption_key_id, + encryption_secret_key: &encryption_secret_key, + signing_key_id, + signature, + }, + recipient, + recipient_key_id, + request, + esk_seed, + platform_version, + ); + encryption_secret_key.non_secure_erase(); + Ok(sealed?) + } +} + +#[cfg(test)] +impl SeedCryptoProvider { + /// The secret key at `path` of the test seed. + fn secret_key_at( + &self, + path: &key_wallet::bip32::DerivationPath, + ) -> Result { + let xprv = self + .wallet + .derive_extended_private_key(path) + .map_err(|e| PlatformWalletError::InvalidIdentityData(format!("test derive: {e}")))?; + dpp::secp256k1::SecretKey::from_secret_bytes(*xprv.private_key.to_bytes()) + .map_err(|e| PlatformWalletError::KeyDerivation(e.to_string())) + } } #[cfg(test)] @@ -378,6 +564,98 @@ pub enum AutoAcceptProofSource { }, } +/// What sending a reply to a contact request came to. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(super) enum ReplyOutcome { + /// The reply was sent. + Sent, + /// A reply to the request's sender was already tracked by the time the + /// pair was claimed or reserved: nothing was sent, and it is adopted. + AlreadyReplied, +} + +/// The claim a public request holds on its pair from before its channel is +/// decided until it is submitted and recorded +/// (`DashPayView::claim_public_pair`): while it is held no private request to +/// the recipient is reserved. Released by [`Self::release`] or on drop. +pub(super) struct PublicContactClaim { + wallet_manager: Arc>>, + wallet_id: WalletId, + sender: Identifier, + recipient: Identifier, + /// The attempt the claim was taken under: only it releases the claim. + attempt: u64, + held: bool, +} + +impl PublicContactClaim { + /// Release the claim now. Dropped while it waits for the wallet lock, + /// the guard releases it. + pub(super) async fn release(mut self) { + let mut wm = self.wallet_manager.write().await; + release_public_claim( + &mut wm, + &self.wallet_id, + &self.sender, + &self.recipient, + self.attempt, + ); + self.held = false; + } +} + +impl Drop for PublicContactClaim { + fn drop(&mut self) { + if !self.held { + return; + } + let (wallet_id, sender, recipient, attempt) = + (self.wallet_id, self.sender, self.recipient, self.attempt); + apply_detached( + &self.wallet_manager, + "a dropped public contact request's claim", + move |wm| release_public_claim(wm, &wallet_id, &sender, &recipient, attempt), + ); + } +} + +/// Release the claim `attempt` holds on `sender`'s pair with `recipient`. +fn release_public_claim( + wm: &mut WalletManager, + wallet_id: &WalletId, + sender: &Identifier, + recipient: &Identifier, + attempt: u64, +) { + if let Some(managed) = wm + .get_wallet_info_mut(wallet_id) + .and_then(|info| info.identity_manager.managed_identity_mut(sender)) + { + managed.release_public_contact_request(recipient, attempt); + } +} + +/// What a contact request to send is built from, whether it is published as a +/// `contactRequest` document or sealed privately: see +/// `DashPayView::prepare_contact_request`. +pub(super) struct PreparedContactRequest { + pub(super) sender_identity: Identity, + pub(super) sender_key_index: u32, + /// The path of the sender's encryption key, whose scalar keys the ECDH and + /// the account reference mask in the signer. + pub(super) sender_enc_path: key_wallet::bip32::DerivationPath, + pub(super) recipient_identity: Identity, + pub(super) recipient_key_index: u32, + pub(super) recipient_enc_pubkey: dpp::secp256k1::PublicKey, + pub(super) account_index: u32, + /// Our receiving (friendship) xpub for the pair. + pub(super) contact_xpub_ext: key_wallet::bip32::ExtendedPubKey, + /// Its DIP-15 69-byte compact form, which the request carries. + pub(super) xpub_bytes: Vec, + pub(super) account_reference: u32, + pub(super) auto_accept_proof: Option>, +} + impl AutoAcceptProofSource { /// Map a pre-built optional proof (the FFI / legacy shape) into a source. pub fn from_option(proof: Option>) -> Self { @@ -415,6 +693,21 @@ impl DashPayView<'_, B> { /// for seedless / external-signable wallets: the raw ECDH scalar stays in /// the signer and only the (public) xpub, the shared secret, and the masked /// reference cross back. + /// + /// A recipient whose requests with this identity came or went privately is + /// refused ([`PlatformWalletError::PrivateContactRequestNeedsPrivateReply`]). + /// So is one a private request is pending to + /// ([`PlatformWalletError::PrivateContactRequestPending`]), until its + /// payment fails for good or a private sync reads it back: it may already + /// be on chain. Before the pair is claimed, DashPay's private store is + /// read to its end when it was not since load, so a request sent + /// privately in an earlier process or on another device is tracked first; + /// while that read cannot finish the send is refused with + /// [`PlatformWalletError::PrivateContactRequestPending`], and on a network + /// that serves no private documents yet nothing is read. A request whose + /// payment is unconfirmed since before a restart holds the pair from the + /// moment the wallet binds to its shielded coordinator + /// (`PlatformWallet::restore_private_contact_payments`). #[allow(clippy::type_complexity)] pub async fn send_contact_request_with_external_signer( &self, @@ -428,6 +721,328 @@ impl DashPayView<'_, B> { where S: Signer + Send + Sync, C: ContactCryptoProvider + Sync, + { + // The pair is claimed before its channel is decided and held until + // the request is submitted and recorded, so a private request cannot + // be reserved and paid for in between; the claim is released when + // the send ends, however it ends. + self.send_public_contact_request( + sender_identity_id, + recipient_identity_id, + account_label, + auto_accept_proof, + signer, + crypto, + false, + ) + .await? + .ok_or_else(|| { + PlatformWalletError::InvalidParameter( + "no contact request was sent: only a reply adopts one already sent".to_string(), + ) + }) + } + + /// [`Self::send_contact_request_with_external_signer`], for a reply when + /// `adopt_reply`: then a request to the recipient tracked by the time + /// the pair is claimed (another accept's reply) is adopted rather than + /// rotated past, and `None` comes back with nothing sent. + #[allow(clippy::too_many_arguments)] + pub(super) async fn send_public_contact_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + signer: &S, + crypto: &C, + adopt_reply: bool, + ) -> Result, PlatformWalletError> + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + { + // Bound to this handle's wallet generation, under its lifecycle gate + // from before the channel is decided through the submission and its + // bookkeeping: a removal cannot complete in between, and a send + // started on a wallet since removed and re-created under the same id + // publishes nothing and records nothing on the replacement. + let _lifecycle = self.generation.payment_guard().await; + let live = self + .wallet_manager + .read() + .await + .get_wallet_info(&self.wallet_id) + .is_some_and(|info| Arc::ptr_eq(&info.generation, &self.generation)); + if !live { + return Err(PlatformWalletError::WalletRemovedBeforeSend(hex::encode( + self.wallet_id, + ))); + } + self.read_private_store_before_deciding(sender_identity_id, recipient_identity_id, crypto) + .await?; + self.release_discharged_private_requests().await?; + let Some(claim) = self + .claim_public_pair(sender_identity_id, recipient_identity_id, adopt_reply) + .await? + else { + return Ok(None); + }; + let sent = self + .send_claimed_contact_request( + sender_identity_id, + recipient_identity_id, + account_label, + auto_accept_proof, + signer, + crypto, + ) + .await; + claim.release().await; + sent.map(Some) + } + + /// Claim the pair `sender`, `recipient` for a public request: refused, + /// in the same lock, while a private request to the recipient is + /// reserved, or once their requests came or went privately, as a + /// `contactRequest` document would publish the relationship (replies and + /// rotations go privately instead). The refusal reads what the wallet + /// tracks, so on a restored or second device it holds only once the + /// private sync has read the store. + pub(super) async fn claim_public_pair( + &self, + sender: &Identifier, + recipient: &Identifier, + adopt_reply: bool, + ) -> Result, PlatformWalletError> { + let mut wm = self.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))? + .identity_manager + .managed_identity_mut(sender) + .ok_or(PlatformWalletError::IdentityNotFound(*sender))?; + // A reply finds the one another accept recorded, in the same lock. + if adopt_reply && managed.dashpay().outgoing_request(recipient).is_some() { + return Ok(None); + } + if managed.dashpay().private_reply_pending(recipient) + || self + .persister + .absent_sender_private_pair_held(sender, recipient) + { + return Err(PlatformWalletError::PrivateContactRequestPending( + *recipient, + )); + } + if managed.dashpay().private_contact(recipient).is_some() { + return Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply( + *recipient, + )); + } + let attempt = managed.claim_public_contact_request(*recipient); + Ok(Some(PublicContactClaim { + wallet_manager: Arc::clone(&self.wallet_manager), + wallet_id: self.wallet_id, + sender: *sender, + recipient: *recipient, + attempt, + held: true, + })) + } + + /// The body of [`Self::send_contact_request_with_external_signer`], once + /// the pair is claimed. + async fn send_claimed_contact_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + signer: &S, + crypto: &C, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + { + // 1-4c. The keys, the receiving xpub, the masked account reference and + // the auto-accept proof, as a private request is built from them too. + let PreparedContactRequest { + sender_identity, + sender_key_index, + sender_enc_path, + recipient_identity, + recipient_key_index, + recipient_enc_pubkey, + account_index, + contact_xpub_ext, + xpub_bytes, + account_reference, + auto_accept_proof, + } = self + .prepare_contact_request( + sender_identity_id, + recipient_identity_id, + auto_accept_proof, + crypto, + ) + .await?; + + // 5. Build the signing key reference for document signing. + let identity_public_key = sender_identity + // Contact-request send writes a document state transition, + // which DPP requires to be signed by a HIGH-or-stricter + // authentication key. MASTER is rejected on document writes. + .available_signing_key( + signer, + Purpose::AUTHENTICATION, + &[SecurityLevel::HIGH, SecurityLevel::CRITICAL], + &[KeyType::ECDSA_SECP256K1], + false, + )? + .cloned() + .ok_or_else(|| { + PlatformWalletError::InvalidIdentityData( + "Sender identity has no HIGH or CRITICAL authentication key \ + (required for document state transitions)" + .to_string(), + ) + })?; + + // 6. Client-side ECDH via the signer: the shared secret is derived in + // the signer (scalar at `sender_enc_path`) against the recipient's + // encryption key, so the SDK write helper receives the finished secret + // (`EcdhProvider::ClientSide`) and no private key is ever materialized + // here. The recipient key is resolved exactly as the SDK would + // (`recipientKeyIndex` on the recipient identity); the helper re-checks + // the SDK asks for this same key before using the secret. + let shared_secret = crypto + .ecdh_shared_secret(&sender_enc_path, &recipient_enc_pubkey) + .await?; + + // 7. Broadcast through the write helper. All inputs are resolved + // above; the helper assembles the SDK `EcdhProvider` + xpub + // closure and dispatches `Sdk::send_contact_request`, keeping + // the seven-generic SDK signature out of this call site — see + // `sdk_writer.rs`. + let result = self + .sdk_writer + .send_contact_request(SendContactRequestParams { + sender_identity: sender_identity.clone(), + recipient_identity, + sender_key_index, + recipient_key_index, + account_reference, + account_label, + auto_accept_proof, + shared_secret, + expected_recipient_pubkey: recipient_enc_pubkey, + xpub_bytes, + signing_public_key: identity_public_key, + signer: signer as &(dyn Signer + Send + Sync), + }) + .await?; + + // 8. Mirror the local-state bookkeeping in `send_contact_request`. + // + // Store the REAL 96-byte ciphertext off the broadcast + // document (not a zero placeholder) so the persisted / + // SwiftData row matches what landed on Platform — a restored + // device comparing local rows against chain sees identity, + // and the sent-side re-ingest doesn't "upgrade" the row. + // Hard error rather than a zero-fill fallback: persisting a 96-byte + // all-zero "valid-looking" ciphertext would poison the local row + // (a restored device compares it to chain and mismatches; anything + // treating it as the contact's xpub source decrypts garbage). The + // broadcast already landed on-chain, so the sweep re-ingests + // the real document on the next pass — returning an error here is + // strictly safer than silently storing poison in release builds. + let encrypted_public_key = result + .document + .properties() + .get("encryptedPublicKey") + .and_then(|v: &Value| v.to_binary_bytes().ok()) + .ok_or_else(|| { + PlatformWalletError::InvalidIdentityData( + "broadcast contactRequest lacks a readable encryptedPublicKey; \ + the on-chain doc will reconcile on the next sync" + .to_string(), + ) + })?; + let mut contact_request = ContactRequest::new( + *sender_identity_id, + result.recipient_id, + sender_key_index, + recipient_key_index, + result.account_reference, + encrypted_public_key, + result.document.created_at_core_block_height().unwrap_or(0), + result.document.created_at().unwrap_or(0), + ); + // Mirror the broadcast doc's optional `encryptedAccountLabel` onto the + // local outgoing row so it matches what landed on Platform (same reason + // the 96-byte ciphertext is read off the doc above). Without this the + // sender's own row never reflects the label it just sent. + contact_request.encrypted_account_label = result + .document + .properties() + .get("encryptedAccountLabel") + .and_then(|v: &Value| v.to_binary_bytes().ok()); + + { + let mut wm = self.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + let managed = info + .identity_manager + .managed_identity_mut(sender_identity_id) + .ok_or(PlatformWalletError::IdentityNotFound(*sender_identity_id))?; + managed + .add_sent_contact_request(contact_request.clone(), &self.persister) + .map_err(|e| { + PlatformWalletError::Persistence(format!( + "sent contact request not persisted: {e}" + )) + })?; + } + + // Register our receiving (friendship) account from the xpub the signer + // already derived above — same friendship key, no second derivation and + // no resident seed. + self.register_contact_account( + sender_identity_id, + recipient_identity_id, + account_index, + contact_xpub_ext, + ) + .await?; + + Ok(contact_request) + } + + /// Steps 1 to 4c of a contact request send, shared by the document send + /// ([`Self::send_contact_request_with_external_signer`]) and the private one + /// (`send_private_contact_request`): the sender's identity, encryption key + /// and its path, the recipient's identity and key, our receiving xpub for + /// the pair, the DIP-15 masked account reference (its version bumped past + /// the previous request to the recipient, public or private) and the + /// auto-accept proof over it. + /// + /// Once the version is at its last, 15, a request with the same xpub and + /// key would repeat the previous one: it is refused with + /// [`PlatformWalletError::InvalidParameter`], before anything is signed, + /// sealed or paid. + pub(super) async fn prepare_contact_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + auto_accept_proof: AutoAcceptProofSource, + crypto: &C, + ) -> Result + where + C: ContactCryptoProvider + Sync, { // 1. Retrieve the sender identity and its HD index from the // local manager. @@ -558,17 +1173,17 @@ impl DashPayView<'_, B> { // identical resend, so the bump is what makes a superseding // (rotation) request broadcastable. The HMAC+mask runs in the // signer (keyed by the raw scalar at `sender_enc_path`). + let prior_reference = { + let wm = self.wallet_manager.read().await; + wm.get_wallet_info(&self.wallet_id) + .and_then(|info| info.identity_manager.managed_identity(sender_identity_id)) + // Checks both the pending sent map AND the established + // contact's outgoing request: see the method doc for + // why consulting only the pending map breaks rotation + // on established contacts. + .and_then(|managed| managed.prior_sent_account_reference(recipient_identity_id)) + }; let account_reference = { - let prior_reference = { - let wm = self.wallet_manager.read().await; - wm.get_wallet_info(&self.wallet_id) - .and_then(|info| info.identity_manager.managed_identity(sender_identity_id)) - // Checks both the pending sent map AND the established - // contact's outgoing request — see the method doc for - // why consulting only the pending map breaks rotation - // on established contacts. - .and_then(|managed| managed.prior_sent_account_reference(recipient_identity_id)) - }; let previous_version = match prior_reference { Some(prior) => Some( crypto @@ -580,188 +1195,80 @@ impl DashPayView<'_, B> { }; let version = match previous_version { // 4-bit field; saturate rather than wrap so a 16th - // rotation fails loudly at the unique index instead of - // silently colliding with version 0. + // rotation is refused below instead of silently colliding + // with version 0. Some(v) if v >= 15 => { tracing::warn!( - recipient = %recipient_identity_id, - "accountReference rotation version saturated at 15" - ); - 15 - } - Some(v) => v + 1, - None => 0, - }; - crypto - .account_reference(&sender_enc_path, &xpub_bytes, account_index, version) - .await? - }; - - // 4c. Resolve the auto-accept proof now that `account_reference` is - // known. The QR-scanner variant signs `(sender, recipient, - // account_reference)` here with the handed key, binding the proof to - // the exact reference this document carries. - let auto_accept_proof: Option> = match auto_accept_proof { - AutoAcceptProofSource::None => None, - AutoAcceptProofSource::Provided(p) => Some(p), - AutoAcceptProofSource::SignWithKey { secret_key, expiry } => Some( - crate::wallet::identity::crypto::auto_accept::sign_auto_accept_proof( - &secret_key, - sender_identity_id, - recipient_identity_id, - account_reference, - expiry, - ), - ), - }; - - // 5. Build the signing key reference for document signing. - let identity_public_key = sender_identity - // Contact-request send writes a document state transition, - // which DPP requires to be signed by a HIGH-or-stricter - // authentication key. MASTER is rejected on document writes. - .available_signing_key( - signer, - Purpose::AUTHENTICATION, - &[SecurityLevel::HIGH, SecurityLevel::CRITICAL], - &[KeyType::ECDSA_SECP256K1], - false, - )? - .cloned() - .ok_or_else(|| { - PlatformWalletError::InvalidIdentityData( - "Sender identity has no HIGH or CRITICAL authentication key \ - (required for document state transitions)" - .to_string(), - ) - })?; - - // 6. Client-side ECDH via the signer: the shared secret is derived in - // the signer (scalar at `sender_enc_path`) against the recipient's - // encryption key, so the SDK write helper receives the finished secret - // (`EcdhProvider::ClientSide`) and no private key is ever materialized - // here. The recipient key is resolved exactly as the SDK would - // (`recipientKeyIndex` on the recipient identity); the helper re-checks - // the SDK asks for this same key before using the secret. - let recipient_enc_pubkey = { - use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; - let key = recipient_identity - .public_keys() - .get(&recipient_key_index) - .ok_or_else(|| { - PlatformWalletError::InvalidIdentityData(format!( - "Recipient identity has no key at index {recipient_key_index}" - )) - })?; - dpp::secp256k1::PublicKey::from_slice(key.data().as_slice()).map_err(|e| { - PlatformWalletError::InvalidIdentityData(format!( - "Recipient encryption public key is invalid: {e}" - )) - })? - }; - let shared_secret = crypto - .ecdh_shared_secret(&sender_enc_path, &recipient_enc_pubkey) - .await?; - - // 7. Broadcast through the write helper. All inputs are resolved - // above; the helper assembles the SDK `EcdhProvider` + xpub - // closure and dispatches `Sdk::send_contact_request`, keeping - // the seven-generic SDK signature out of this call site — see - // `sdk_writer.rs`. - let result = self - .sdk_writer - .send_contact_request(SendContactRequestParams { - sender_identity: sender_identity.clone(), - recipient_identity, - sender_key_index, - recipient_key_index, - account_reference, - account_label, - auto_accept_proof, - shared_secret, - expected_recipient_pubkey: recipient_enc_pubkey, - xpub_bytes, - signing_public_key: identity_public_key, - signer: signer as &(dyn Signer + Send + Sync), - }) - .await?; - - // 8. Mirror the local-state bookkeeping in `send_contact_request`. - // - // Store the REAL 96-byte ciphertext off the broadcast - // document (not a zero placeholder) so the persisted / - // SwiftData row matches what landed on Platform — a restored - // device comparing local rows against chain sees identity, - // and the sent-side re-ingest doesn't "upgrade" the row. - // Hard error rather than a zero-fill fallback: persisting a 96-byte - // all-zero "valid-looking" ciphertext would poison the local row - // (a restored device compares it to chain and mismatches; anything - // treating it as the contact's xpub source decrypts garbage). The - // broadcast already landed on-chain, so the sweep re-ingests - // the real document on the next pass — returning an error here is - // strictly safer than silently storing poison in release builds. - let encrypted_public_key = result - .document - .properties() - .get("encryptedPublicKey") - .and_then(|v: &Value| v.to_binary_bytes().ok()) - .ok_or_else(|| { - PlatformWalletError::InvalidIdentityData( - "broadcast contactRequest lacks a readable encryptedPublicKey; \ - the on-chain doc will reconcile on the next sync" - .to_string(), - ) - })?; - let mut contact_request = ContactRequest::new( - *sender_identity_id, - result.recipient_id, - sender_key_index, - recipient_key_index, - result.account_reference, - encrypted_public_key, - result.document.created_at_core_block_height().unwrap_or(0), - result.document.created_at().unwrap_or(0), - ); - // Mirror the broadcast doc's optional `encryptedAccountLabel` onto the - // local outgoing row so it matches what landed on Platform (same reason - // the 96-byte ciphertext is read off the doc above). Without this the - // sender's own row never reflects the label it just sent. - contact_request.encrypted_account_label = result - .document - .properties() - .get("encryptedAccountLabel") - .and_then(|v: &Value| v.to_binary_bytes().ok()); + recipient = %recipient_identity_id, + "accountReference rotation version saturated at 15" + ); + 15 + } + Some(v) => v + 1, + None => 0, + }; + crypto + .account_reference(&sender_enc_path, &xpub_bytes, account_index, version) + .await? + }; + // Saturated, with the same xpub and key: the request would repeat the + // tracked one. A document send would fail at the unique index; a + // private one would be paid for and then ignored as already tracked. + if prior_reference == Some(account_reference) { + return Err(PlatformWalletError::InvalidParameter(format!( + "no account reference version is left for another contact request to \ + {recipient_identity_id}: version 15 is used, and this request would repeat the \ + one already sent" + ))); + } - { - let mut wm = self.wallet_manager.write().await; - let info = wm - .get_wallet_info_mut(&self.wallet_id) - .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; - let managed = info - .identity_manager - .managed_identity_mut(sender_identity_id) - .ok_or(PlatformWalletError::IdentityNotFound(*sender_identity_id))?; - managed - .add_sent_contact_request(contact_request.clone(), &self.persister) - .map_err(|e| { - PlatformWalletError::Persistence(format!( - "sent contact request not persisted: {e}" + // 4c. Resolve the auto-accept proof now that `account_reference` is + // known. The QR-scanner variant signs `(sender, recipient, + // account_reference)` here with the handed key, binding the proof to + // the exact reference this document carries. + let auto_accept_proof: Option> = match auto_accept_proof { + AutoAcceptProofSource::None => None, + AutoAcceptProofSource::Provided(p) => Some(p), + AutoAcceptProofSource::SignWithKey { secret_key, expiry } => Some( + crate::wallet::identity::crypto::auto_accept::sign_auto_accept_proof( + &secret_key, + sender_identity_id, + recipient_identity_id, + account_reference, + expiry, + ), + ), + }; + + let recipient_enc_pubkey = { + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + let key = recipient_identity + .public_keys() + .get(&recipient_key_index) + .ok_or_else(|| { + PlatformWalletError::InvalidIdentityData(format!( + "Recipient identity has no key at index {recipient_key_index}" )) })?; - } - - // Register our receiving (friendship) account from the xpub the signer - // already derived above — same friendship key, no second derivation and - // no resident seed. - self.register_contact_account( - sender_identity_id, - recipient_identity_id, + dpp::secp256k1::PublicKey::from_slice(key.data().as_slice()).map_err(|e| { + PlatformWalletError::InvalidIdentityData(format!( + "Recipient encryption public key is invalid: {e}" + )) + })? + }; + Ok(PreparedContactRequest { + sender_identity, + sender_key_index, + sender_enc_path, + recipient_identity, + recipient_key_index, + recipient_enc_pubkey, account_index, contact_xpub_ext, - ) - .await?; - - Ok(contact_request) + xpub_bytes, + account_reference, + auto_accept_proof, + }) } } @@ -826,10 +1333,11 @@ fn query_lower_bound(high_water: Option) -> Option { /// Collapse a stream of parsed received contact requests to the single /// newest request per sender, keyed by `sender_id`. /// -/// "Newest" is the lexicographic max of `(created_at, account_reference)` -/// — created_at is the primary signal (a rotation request is broadcast -/// later), with account_reference as a deterministic tiebreak for the -/// degenerate same-timestamp case. +/// "Newest" is the lexicographic max of the account reference's version, +/// `created_at`, then `account_reference` ([`freshness`]): a rotation bumps +/// the version and is broadcast later, and a replay of an older request +/// appended again later keeps its older version. The reference is a +/// deterministic tiebreak for the degenerate same-timestamp case. /// /// This is the idempotency keystone of the recurring sync: on-chain /// `contactRequest` docs are immutable and never deleted, so a sender who @@ -861,7 +1369,7 @@ fn query_lower_bound(high_water: Option) -> Option { /// /// DIP-15 section 8.4 allows ignoring additional requests, which is what the /// collapse does. -fn newest_received_per_sender( +pub(super) fn newest_received_per_sender( requests: impl IntoIterator, ) -> std::collections::BTreeMap { let mut newest: std::collections::BTreeMap = @@ -870,9 +1378,7 @@ fn newest_received_per_sender( let sender = req.sender_id; let replace = newest .get(&sender) - .map(|cur| { - (req.created_at, req.account_reference) > (cur.created_at, cur.account_reference) - }) + .map(|cur| freshness(&req) > freshness(cur)) .unwrap_or(true); if replace { newest.insert(sender, req); @@ -894,7 +1400,7 @@ fn newest_received_per_sender( /// with the freshest outgoing reference — otherwise the next rotation /// collides on the contract's `($ownerId, toUserId, accountReference)` unique /// index. -fn newest_sent_per_recipient( +pub(super) fn newest_sent_per_recipient( requests: impl IntoIterator, ) -> std::collections::BTreeMap { let mut newest: std::collections::BTreeMap = @@ -903,9 +1409,7 @@ fn newest_sent_per_recipient( let recipient = req.recipient_id; let replace = newest .get(&recipient) - .map(|cur| { - (req.created_at, req.account_reference) > (cur.created_at, cur.account_reference) - }) + .map(|cur| freshness(&req) > freshness(cur)) .unwrap_or(true); if replace { newest.insert(recipient, req); @@ -929,7 +1433,13 @@ pub(super) fn ingest_sent_sweep( fetch_ok: bool, ) -> bool { let newest_by_recipient = record_and_collapse_sent_requests(managed, requests); - let persisted = ingest_sent_requests(managed, persister, identity_id, newest_by_recipient); + let persisted = ingest_sent_requests( + managed, + persister, + identity_id, + newest_by_recipient, + &mut Vec::new(), + ); if fetch_ok && persisted { managed.mark_sent_sweep_completed(); } @@ -963,6 +1473,38 @@ fn record_and_collapse_sent_requests( newest_sent_per_recipient(requests) } +/// Whether `candidate` is older than `tracked`, a request between the same +/// pair: by its account reference's version, which a rotation bumps, then by +/// its creation time. A replay of an older request appended again later has a +/// newer time but an older version. +pub(super) fn predates(candidate: &ContactRequest, tracked: &ContactRequest) -> bool { + freshness(candidate) < freshness(tracked) +} + +/// Whether `tracked`, a request between one pair, is an unconfirmed request +/// carrying `account_reference` or supersedes it in the order [`predates`] +/// reads: the same reference, or a later version. The unconfirmed request is +/// not in the store yet, so it has no creation time; appended, it would be +/// the newest of its version. +pub(super) fn records_or_supersedes(tracked: &ContactRequest, account_reference: u32) -> bool { + tracked.account_reference == account_reference + || platform_encryption::account_reference_version(account_reference) + < platform_encryption::account_reference_version(tracked.account_reference) +} + +/// How fresh a request between one pair is, for collapsing a sweep's requests +/// and for passing over older ones alike: by its account reference's version, +/// which a rotation bumps, then by its creation time, then by the reference +/// itself so that equal times order deterministically. A replay of an older +/// request appended again later has a newer time but an older version. +fn freshness(request: &ContactRequest) -> (u32, u64, u32) { + ( + platform_encryption::account_reference_version(request.account_reference), + request.created_at, + request.account_reference, + ) +} + /// Ingest one identity's collapsed **received** contact requests into local /// state, returning whether every write reached disk. /// @@ -978,7 +1520,7 @@ fn record_and_collapse_sent_requests( /// /// Split out of the sweep so the persist-failure branches are reachable in a /// test without standing up a Platform that answers document queries. -fn ingest_received_requests( +pub(super) fn ingest_received_requests( managed: &mut crate::wallet::identity::ManagedIdentity, persister: &crate::wallet::persister::WalletPersister, identity_id: Identifier, @@ -1008,21 +1550,18 @@ fn ingest_received_requests( // tracked incoming/established state is a re-ingest of a known doc; a // DIFFERENT reference from a known sender is a rotation request // (receive side) and must get through. - let tracked_reference = managed - .dashpay() - .incoming_contact_requests() - .get(&sender_id) - .map(|r| r.account_reference) - .or_else(|| { - managed - .dashpay() - .established_contacts() - .get(&sender_id) - .map(|c| c.incoming_request.account_reference) - }); + let tracked = managed.dashpay().incoming_request(&sender_id); + let tracked_reference = tracked.map(|r| r.account_reference); if tracked_reference == Some(contact_request.account_reference) { continue; } + // An older request than the tracked one is no rotation: one channel's + // sweep can replay a request the other channel already superseded (a + // cold start scans the private store from its start), and taking it + // would roll the contact back to a retired xpub. + if tracked.is_some_and(|tracked| predates(&contact_request, tracked)) { + continue; + } if tracked_reference.is_some() { // Rotation: supersede the tracked request. When an established @@ -1062,20 +1601,43 @@ fn ingest_received_requests( /// /// `add_sent_contact_request` carries its own duplicate / metadata-loss guard, /// so re-ingesting the same range on the next sweep is safe. -fn ingest_sent_requests( +/// +/// `recorded` gets each request this recorded: not one already tracked with +/// the same account reference, a replay, or one whose store failed. +pub(super) fn ingest_sent_requests( managed: &mut ManagedIdentity, persister: &WalletPersister, identity_id: Identifier, newest_by_recipient: std::collections::BTreeMap, + recorded: &mut Vec, ) -> bool { - for (_recipient_id, contact_request) in newest_by_recipient { - if let Err(e) = managed.add_sent_contact_request(contact_request, persister) { + for (recipient_id, contact_request) in newest_by_recipient { + // An older request than the tracked one is a replay, as on the + // received side: superseding with it would re-derive the next rotation + // from a retired reference. + let (replay, tracked) = + managed + .dashpay() + .outgoing_request(&recipient_id) + .map_or((false, false), |tracked| { + ( + predates(&contact_request, tracked), + tracked.account_reference == contact_request.account_reference, + ) + }); + if replay { + continue; + } + if let Err(e) = managed.add_sent_contact_request(contact_request.clone(), persister) { tracing::error!( owner = %identity_id, error = %e, "sent-request ingest persist failed; leaving sent cursor for retry" ); return false; } + if !tracked { + recorded.push(contact_request); + } } true } @@ -1149,6 +1711,61 @@ pub(crate) fn budget_spent(deadline: Option) -> bool { deadline.is_some_and(|d| std::time::Instant::now() >= d) } +/// Tears down the external (sending) accounts of `identity_id`'s contacts that +/// must be rebuilt: those `rotated` this pass, re-keyed by a rotation request, +/// and any registered account not built from its contact's current incoming +/// request. Removes both the immutable `Account` (old xpub, `send_payment`'s +/// derivation source) and the managed wrapper (old address pool); the account +/// build sweep then re-queues them and re-registers from the current xpub, and +/// the persisted account row is upserted (same unique key) when that round +/// lands. +/// +/// The second case is the rotation self-heal across a restart: an external +/// account rebuilt from the persisted (tombstone-less) registration row can +/// carry the STALE xpub while the established contact already tracks the new +/// incoming reference (the deferred rebuild was lost with the in-memory queue +/// at load). Such an account exists but was not built from the current +/// reference, so the plain "has an external account" gate would skip it +/// forever: a recorded `external_account_reference` that does not match the +/// contact's current `incoming_request.account_reference`, including a `None` +/// marker from a cold restore that did not carry it, queues it here. +/// Idempotent: once rebuilt the marker is stamped and the next sweep skips it. +pub(super) fn tear_down_stale_external_accounts( + wallet: &mut key_wallet::wallet::Wallet, + info: &mut PlatformWalletInfo, + identity_id: &Identifier, + mut rotated: Vec, +) { + use key_wallet::account::account_collection::DashpayAccountKey; + let key_for = |contact_id: &Identifier| DashpayAccountKey { + index: 0, + user_identity_id: identity_id.to_buffer(), + friend_identity_id: contact_id.to_buffer(), + }; + if let Some(managed) = info.identity_manager.managed_identity(identity_id) { + for (contact_id, contact) in managed.dashpay().established_contacts().iter() { + let has_external = info + .core_wallet + .accounts + .dashpay_external_accounts + .contains_key(&key_for(contact_id)); + if external_account_needs_rebuild(contact, has_external) + && !rotated.contains(contact_id) + { + rotated.push(*contact_id); + } + } + } + for contact_id in &rotated { + let key = key_for(contact_id); + wallet.accounts.dashpay_external_accounts.remove(&key); + info.core_wallet + .accounts + .dashpay_external_accounts + .remove(&key); + } +} + /// Whether a registered outbound `DashpayExternalAccount` for `contact` /// must be torn down + rebuilt because it was NOT built from the contact's /// current `incoming_request.account_reference`. @@ -1623,69 +2240,6 @@ impl DashPayView<'_, B> { let sent_persist_ok = ingest_sent_sweep(managed, &self.persister, identity_id, parsed_sent, sent_ok); - // (2a') Rotation self-heal across restart: an external account - // rebuilt from the persisted (tombstone-less) registration - // row after a restart can carry the STALE xpub while the - // established contact already tracks the new incoming - // reference (the deferred rebuild was lost with the - // in-memory queue at load). Such an account exists but was - // NOT built from the current reference, so the plain - // `has_external` gate would skip it forever. Detect a - // registered external account whose recorded - // `external_account_reference` does not match the contact's - // current `incoming_request.account_reference` (including a - // `None` marker from a cold restore that didn't carry it), - // and enqueue it for teardown + rebuild alongside the - // in-pass rotations. Idempotent: once rebuilt the marker is - // stamped, so the next sweep sees a match and skips it. - // Accesses the managed identity's established contacts and - // `info.core_wallet` as DISJOINT fields of `info` via a - // plain `for` loop (the same split the teardown loop below - // relies on) — a closure capturing whole `info` would - // collide with the mutable `managed` reborrow. - { - use key_wallet::account::account_collection::DashpayAccountKey; - for (contact_id, contact) in managed.dashpay().established_contacts().iter() { - let key = DashpayAccountKey { - index: 0, - user_identity_id: identity_id.to_buffer(), - friend_identity_id: contact_id.to_buffer(), - }; - let has_external = info - .core_wallet - .accounts - .dashpay_external_accounts - .contains_key(&key); - if external_account_needs_rebuild(contact, has_external) - && !rotated_contacts.contains(contact_id) - { - rotated_contacts.push(*contact_id); - } - } - } - - // (2b) Tear down stale external accounts for contacts that - // rotated in this pass: both the immutable Account - // (old xpub — `send_payment`'s derivation source) and - // the managed wrapper (old address pool). The - // candidate collection below then re-queues them and - // the build step re-registers from the NEW encrypted - // xpub. The persisted account row is upserted (same - // unique key) when the re-registration round lands. - for contact_id in &rotated_contacts { - use key_wallet::account::account_collection::DashpayAccountKey; - let key = DashpayAccountKey { - index: 0, - user_identity_id: identity_id.to_buffer(), - friend_identity_id: contact_id.to_buffer(), - }; - wallet.accounts.dashpay_external_accounts.remove(&key); - info.core_wallet - .accounts - .dashpay_external_accounts - .remove(&key); - } - // Advance the high-water cursors to the max `$createdAt` // fetched this sweep, never below the current value. Advance a // direction's cursor only when BOTH its fetch succeeded AND @@ -1716,6 +2270,11 @@ impl DashPayView<'_, B> { report.unpersisted_identities.push(identity_id); } + // (2a', 2b) Tear down the external accounts of contacts re-keyed + // this pass, or rebuilt after a restart from a stale + // xpub, so the build below re-registers them. + tear_down_stale_external_accounts(wallet, info, &identity_id, rotated_contacts); + // (3) Collect missing receiving accounts for sent-only // requests and account pairs for established contacts. Self::collect_account_build_candidates(info, &identity_id) @@ -2005,7 +2564,7 @@ impl DashPayView<'_, B> { /// /// Called **after** the sync write guard is dropped: this re-acquires /// the non-reentrant wallet-manager lock. - async fn build_contact_accounts( + pub(super) async fn build_contact_accounts( &self, identity_id: &Identifier, candidate: AccountBuildCandidate, @@ -3496,6 +4055,14 @@ impl DashPayView<'_, B> { /// [`Self::send_contact_request_with_external_signer`] so signing /// crosses the FFI via the supplied `&S: Signer`. /// Same ECDH caveat applies — see that method's docstring. + /// + /// A request that came privately is refused with + /// [`PlatformWalletError::PrivateContactRequestNeedsPrivateReply`]: a + /// `contactRequest` document in reply would publish the relationship the + /// sender kept off chain. It is accepted with + /// `accept_private_contact_request`. The store is read first as the send + /// reads it, so the refusal and the reply it adopts see a private reply + /// sent in an earlier process. pub async fn accept_contact_request_with_external_signer( &self, request: &ContactRequest, @@ -3505,13 +4072,81 @@ impl DashPayView<'_, B> { where S: Signer + Send + Sync, C: ContactCryptoProvider + Sync, + { + self.read_private_store_before_deciding(&request.recipient_id, &request.sender_id, crypto) + .await?; + if self.needs_private_reply(request).await { + return Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply( + request.sender_id, + )); + } + self.accept_contact_request_replying_with(request, crypto, || async { + Ok( + match self + .send_public_contact_request( + &request.recipient_id, + &request.sender_id, + None, + AutoAcceptProofSource::None, + signer, + crypto, + true, + ) + .await? + { + Some(_) => ReplyOutcome::Sent, + None => ReplyOutcome::AlreadyReplied, + }, + ) + }) + .await + } + + /// Whether accepting `request` takes a private reply: it, or the request + /// tracked from its sender, came privately, and we have sent the sender + /// no request yet. A reply already sent, either way, is adopted without a + /// broadcast, so only a reply still to send would publish anything. + pub(super) async fn needs_private_reply(&self, request: &ContactRequest) -> bool { + let wm = self.wallet_manager.read().await; + wm.get_wallet_info(&self.wallet_id) + .and_then(|info| { + info.identity_manager + .managed_identity(&request.recipient_id) + }) + .is_some_and(|managed| { + let dashpay = managed.dashpay(); + (request.came_privately() + || dashpay + .incoming_request(&request.sender_id) + .is_some_and(ContactRequest::came_privately)) + && dashpay.outgoing_request(&request.sender_id).is_none() + }) + } + + /// Accept `request`, with `send_reply` sending our reciprocal request to + /// its sender unless one already exists: the body of + /// [`Self::accept_contact_request_with_external_signer`], shared with the + /// private accept, which replies privately. `send_reply` reports a reply + /// tracked by the time it claimed or reserved the pair (another accept's, + /// recorded in between) as [`ReplyOutcome::AlreadyReplied`], and that + /// reply is adopted as one found here first is: nothing is sent or paid. + pub(super) async fn accept_contact_request_replying_with( + &self, + request: &ContactRequest, + crypto: &C, + send_reply: R, + ) -> Result + where + C: ContactCryptoProvider + Sync, + R: FnOnce() -> Fut, + Fut: std::future::Future>, { let our_identity_id = request.recipient_id; let sender_id = request.sender_id; // 1. Verify the incoming request is known, and detect whether an // on-platform reciprocal already exists for this pair. - let (already_established, already_reciprocated) = { + let (_, already_reciprocated) = { let wm = self.wallet_manager.read().await; let info = wm .get_wallet_info(&self.wallet_id) @@ -3559,7 +4194,9 @@ impl DashPayView<'_, B> { // fresh-send local registrations below (receiving account + // validate→decrypt→register external), so the contact becomes // payable without a duplicate broadcast. - if already_reciprocated { + let adopt = + already_reciprocated || matches!(send_reply().await?, ReplyOutcome::AlreadyReplied); + if adopt { tracing::info!( our_identity = %our_identity_id, contact = %sender_id, @@ -3579,19 +4216,27 @@ impl DashPayView<'_, B> { // path the normal flow and the sync sweep use), so the final lookup // is guaranteed non-`None`. Idempotent: `add_incoming_contact_request` // preserves metadata on an already-established pair. - if !already_established { + { let mut wm = self.wallet_manager.write().await; let managed = wm .get_wallet_info_mut(&self.wallet_id) .and_then(|info| info.identity_manager.managed_identity_mut(&our_identity_id)) .ok_or(PlatformWalletError::IdentityNotFound(our_identity_id))?; - managed - .add_incoming_contact_request(request.clone(), &self.persister) - .map_err(|e| { - PlatformWalletError::Persistence(format!( - "accept-adopt: incoming request not persisted: {e}" - )) - })?; + // Read now: a reply adopted after `send_reply` may have + // established the pair since step 1. + if !managed + .dashpay() + .established_contacts() + .contains_key(&sender_id) + { + managed + .add_incoming_contact_request(request.clone(), &self.persister) + .map_err(|e| { + PlatformWalletError::Persistence(format!( + "accept-adopt: incoming request not persisted: {e}" + )) + })?; + } } // Adopt: register the receiving (friendship) account, derived via // the signer (no resident seed), matching the fresh-send path. @@ -3619,16 +4264,6 @@ impl DashPayView<'_, B> { "Accept-adopt: failed to derive receiving xpub via signer; will retry on next sweep" ), } - } else { - self.send_contact_request_with_external_signer( - &our_identity_id, - &sender_id, - None, - AutoAcceptProofSource::None, - signer, - crypto, - ) - .await?; } // 4. Validate key indices (same gate as the sync sweep and the @@ -4043,11 +4678,20 @@ impl DashPayView<'_, B> { // `unignore_sender` removes the sender + rewinds the cursor and // returns the removal changeset (empty if the sender wasn't // ignored). Persist it so the ignored-sender row is deleted. - let cs = managed.unignore_sender(contact_identity_id); + let mut cs = managed.unignore_sender(contact_identity_id); if ::is_empty(&cs) { // Not ignored — nothing to persist, but not an error. return Ok(()); } + // The private read position was rewound with the received cursor: a + // backend that keeps it stores the rewind, so a relaunch reads the + // un-ignored sender's private requests again too. + if self.persister.persists_private_contact_scan_state() { + cs.private_contact_requests_positions.insert( + *identity_id, + managed.private_contact_requests_position_record(), + ); + } self.persister.store(cs.into()).map_err(|e| { PlatformWalletError::Persistence(format!("un-ignore not persisted: {e}")) })?; @@ -4410,6 +5054,137 @@ mod sweep_tests { assert_eq!(managed.dashpay().incoming_contact_requests().len(), 1); } + /// A request older than the tracked one, by its reference's version and + /// then its time, is passed over on both sides: one channel's sweep can + /// replay a request the other already superseded, and a replay appended + /// again later has a newer time but an older version. + #[test] + fn an_older_request_than_the_tracked_one_is_no_rotation() { + let our = 1u8; + let contact = 2u8; + let our_id = Identifier::from([our; 32]); + let contact_id = Identifier::from([contact; 32]); + let mut info = info_with_bare_identity(our); + let managed = info + .identity_manager + .managed_identity_mut(&our_id) + .expect("managed identity"); + let at = |sender: u8, recipient: u8, version: u32, account: u32, created_at: u64| { + let mut request = test_request( + sender, + recipient, + (version << ACCOUNT_REFERENCE_VERSION_SHIFT) | account, + ); + request.created_at = created_at; + request + }; + let tracked_incoming = |managed: &ManagedIdentity| { + managed + .dashpay() + .incoming_request(&contact_id) + .map(|request| request.account_reference) + }; + + let rotated = at(contact, our, 1, 5, 100); + for (case, candidate, expected) in [ + ( + "the rotated request", + rotated.clone(), + rotated.account_reference, + ), + ( + "a replay of the request it rotated past, appended later", + at(contact, our, 0, 4, 200), + rotated.account_reference, + ), + ( + "an earlier request of the same version", + at(contact, our, 1, 6, 50), + rotated.account_reference, + ), + ] { + assert!(ingest_received_requests( + managed, + &noop_persister(), + our_id, + newest_received_per_sender([candidate]), + &mut Vec::new(), + &mut Vec::new(), + )); + assert_eq!(tracked_incoming(managed), Some(expected), "{case}"); + } + let newer = at(contact, our, 2, 7, 300); + assert!(ingest_received_requests( + managed, + &noop_persister(), + our_id, + newest_received_per_sender([newer.clone()]), + &mut Vec::new(), + &mut Vec::new(), + )); + assert_eq!( + tracked_incoming(managed), + Some(newer.account_reference), + "a newer rotation is still taken" + ); + + let sent = at(our, contact, 1, 5, 100); + assert!(ingest_sent_requests( + managed, + &noop_persister(), + our_id, + newest_sent_per_recipient([sent.clone()]), + &mut Vec::new(), + )); + assert!(ingest_sent_requests( + managed, + &noop_persister(), + our_id, + newest_sent_per_recipient([at(our, contact, 0, 4, 200)]), + &mut Vec::new(), + )); + assert_eq!( + managed + .dashpay() + .outgoing_request(&contact_id) + .map(|request| request.account_reference), + Some(sent.account_reference), + "a replay of an older sent request does not supersede the tracked one" + ); + } + + /// Collapsing a sweep's requests orders them by version first, as ingest + /// passes over older ones: a rotation and a replay of the request it + /// rotated past, appended later, collapse to the rotation, both ways. + #[test] + fn a_sweep_collapses_to_the_highest_version_before_the_latest_time() { + let at = |sender: u8, recipient: u8, version: u32, account: u32, created_at: u64| { + let mut request = test_request( + sender, + recipient, + (version << ACCOUNT_REFERENCE_VERSION_SHIFT) | account, + ); + request.created_at = created_at; + request + }; + let rotation = at(2, 1, 1, 5, 200); + let replay = at(2, 1, 0, 4, 300); + assert_eq!( + newest_received_per_sender([rotation.clone(), replay.clone()]) + .get(&Identifier::from([2u8; 32])) + .map(|request| request.account_reference), + Some(rotation.account_reference) + ); + let sent_rotation = at(1, 2, 1, 5, 200); + let sent_replay = at(1, 2, 0, 4, 300); + assert_eq!( + newest_sent_per_recipient([sent_replay, sent_rotation.clone()]) + .get(&Identifier::from([2u8; 32])) + .map(|request| request.account_reference), + Some(sent_rotation.account_reference) + ); + } + /// The first-ingest persist failure (`add_incoming_contact_request`). /// The request is not tracked and must not be reported as newly /// discovered — a caller that took it as real would act on a request that @@ -4502,7 +5277,8 @@ mod sweep_tests { managed, &failing_persister(), our_id, - newest + newest, + &mut Vec::new() )); // Control on the same fixture: the write succeeds, so the ingest @@ -4512,7 +5288,8 @@ mod sweep_tests { managed, &noop_persister(), our_id, - newest + newest, + &mut Vec::new() )); } @@ -4607,6 +5384,11 @@ mod sweep_tests { ), "precondition: the failed write must report the pass incomplete" ); + assert_eq!( + all_requests, + vec![], + "a request whose store failed is not reported" + ); assert!( managed.dashpay().incoming_contact_requests().is_empty(), "a request that never reached disk must not be tracked in memory, or the \ @@ -4721,6 +5503,49 @@ mod sweep_tests { ); } + /// The sent ingest reports what it recorded: a new request, not one whose + /// store failed nor one already tracked. + #[test] + fn should_report_only_what_a_sent_ingest_recorded() { + let our = 1u8; + let our_id = Identifier::from([our; 32]); + let mut info = info_with_bare_identity(our); + let managed = info + .identity_manager + .managed_identity_mut(&our_id) + .expect("managed identity"); + let request = test_request(our, 2, 0); + + let mut recorded = Vec::new(); + assert!(!ingest_sent_requests( + managed, + &failing_persister(), + our_id, + newest_sent_per_recipient([request.clone()]), + &mut recorded, + )); + assert_eq!(recorded, vec![], "a request whose store failed"); + + assert!(ingest_sent_requests( + managed, + &noop_persister(), + our_id, + newest_sent_per_recipient([request.clone()]), + &mut recorded, + )); + assert_eq!(recorded, vec![request.clone()], "a new request"); + + let mut again = Vec::new(); + assert!(ingest_sent_requests( + managed, + &noop_persister(), + our_id, + newest_sent_per_recipient([request]), + &mut again, + )); + assert_eq!(again, vec![], "a request already tracked"); + } + /// Branch 3, the fresh sent request (`add_sent_contact_request`). Its /// same-reference no-op guard returns `Ok(())`, so a memory-committed /// failed write makes the retry report success without storing anything. @@ -4737,7 +5562,13 @@ mod sweep_tests { // Sweep 1: the write fails. let newest = newest_sent_per_recipient([test_request(our, 2, 0)]); assert!( - !ingest_sent_requests(managed, &failing_persister(), our_id, newest), + !ingest_sent_requests( + managed, + &failing_persister(), + our_id, + newest, + &mut Vec::new() + ), "precondition: the failed write must report the pass incomplete" ); assert!( @@ -4750,7 +5581,7 @@ mod sweep_tests { let (persister, backend) = counting_persister(); let newest = newest_sent_per_recipient([test_request(our, 2, 0)]); assert!( - ingest_sent_requests(managed, &persister, our_id, newest), + ingest_sent_requests(managed, &persister, our_id, newest, &mut Vec::new()), "the retry must complete the pass" ); diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/dpns_marketplace.rs b/packages/rs-platform-wallet/src/wallet/identity/network/dpns_marketplace.rs index a448db29490..f54f1791730 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/dpns_marketplace.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/dpns_marketplace.rs @@ -3910,6 +3910,12 @@ mod tests { let wallet_id = wm .create_wallet_with_random_mnemonic(WalletAccountCreationOptions::None) .expect("create wallet"); + // The live generation, as `PlatformWallet::new` takes it. + let generation = Arc::clone( + &wm.get_wallet_info(&wallet_id) + .expect("the wallet's info") + .generation, + ); let wallet_manager = Arc::new(RwLock::new(wm)); let persister = WalletPersister::new(wallet_id, mirror); @@ -3934,6 +3940,7 @@ mod tests { persister, sync_fault: Arc::new(std::sync::atomic::AtomicBool::new(false)), durable_cursors: crate::changeset::DurableCursors::default(), + generation, broadcaster, sdk_writer: Arc::new(super::super::sdk_writer::SdkWriter::new(sdk)), dpns_operation_gate: Arc::new(tokio::sync::Mutex::new(())), diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/identity_handle.rs b/packages/rs-platform-wallet/src/wallet/identity/network/identity_handle.rs index 76634be498f..d252507076f 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/identity_handle.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/identity_handle.rs @@ -45,6 +45,7 @@ use zeroize::Zeroizing; use crate::broadcaster::{SpvBroadcaster, TransactionBroadcaster}; use crate::error::PlatformWalletError; use crate::wallet::asset_lock::manager::AssetLockManager; +use crate::wallet::core::WalletGeneration; use crate::wallet::platform_wallet::{PlatformWalletInfo, WalletId}; /// Default gap limit for identity discovery scanning. @@ -508,6 +509,11 @@ pub struct IdentityWallet { /// adapter and the DashPay rescan reconcile. See /// [`DurableCursors`](crate::changeset::DurableCursors). pub(crate) durable_cursors: crate::changeset::DurableCursors, + /// The wallet generation this handle belongs to: a public contact + /// request is sent only while it is the live one, under its lifecycle + /// gate, so a send started on a wallet since removed and re-created under + /// the same id never publishes or records on the replacement. + pub(crate) generation: Arc, /// Broadcaster for DashPay payment transactions. Distinct from the /// asset-lock broadcaster — the asset-lock manager is always /// `SpvBroadcaster`-pinned, while this one picks the broadcaster @@ -551,6 +557,7 @@ impl Clone for IdentityWallet { persister: self.persister.clone(), sync_fault: Arc::clone(&self.sync_fault), durable_cursors: Arc::clone(&self.durable_cursors), + generation: Arc::clone(&self.generation), broadcaster: Arc::clone(&self.broadcaster), sdk_writer: Arc::clone(&self.sdk_writer), dpns_operation_gate: Arc::clone(&self.dpns_operation_gate), diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/mod.rs b/packages/rs-platform-wallet/src/wallet/identity/network/mod.rs index d431d449039..babb0e9fee1 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/mod.rs @@ -75,6 +75,14 @@ mod payments; pub(crate) use payments::{record_incoming_dashpay_payments, sent_payment_status_for_record}; mod prepared_payment; pub use prepared_payment::PreparedDashPayPayment; +mod private_contact_requests; +#[cfg(feature = "shielded")] +pub(crate) use private_contact_requests::{ + restore_journaled_private_payments, JournaledPrivatePayment, +}; +pub use private_contact_requests::{ + PaymentStarted, PrivateContactReadiness, PrivateContactSyncReport, PrivateDocumentKeyPaths, +}; mod profile; mod public_keys; pub use public_keys::{fetch_identity_public_keys, FetchedIdentityPublicKeys}; diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/payments.rs b/packages/rs-platform-wallet/src/wallet/identity/network/payments.rs index 757b71290a6..5060aa6295d 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/payments.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/payments.rs @@ -13429,6 +13429,7 @@ mod tests { persister: real.persister.clone(), sync_fault: Arc::clone(&real.sync_fault), durable_cursors: Arc::clone(&real.durable_cursors), + generation: Arc::clone(&real.generation), broadcaster: Arc::new(GatedRejectingBroadcaster { entered, release }), sdk_writer: Arc::clone(&real.sdk_writer), dpns_operation_gate: Arc::clone(&real.dpns_operation_gate), @@ -13470,6 +13471,7 @@ mod tests { persister: real.persister.clone(), sync_fault: Arc::clone(&real.sync_fault), durable_cursors: Arc::clone(&real.durable_cursors), + generation: Arc::clone(&real.generation), broadcaster: Arc::new(GatedBroadcaster { entered, release }), sdk_writer: Arc::clone(&real.sdk_writer), dpns_operation_gate: Arc::clone(&real.dpns_operation_gate), @@ -13525,6 +13527,7 @@ mod tests { persister: real.persister.clone(), sync_fault: Arc::clone(&real.sync_fault), durable_cursors: Arc::clone(&real.durable_cursors), + generation: Arc::clone(&real.generation), broadcaster: Arc::new(broadcaster), sdk_writer: Arc::clone(&real.sdk_writer), dpns_operation_gate: Arc::clone(&real.dpns_operation_gate), diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/private_contact_requests.rs b/packages/rs-platform-wallet/src/wallet/identity/network/private_contact_requests.rs new file mode 100644 index 00000000000..ea32bde680c --- /dev/null +++ b/packages/rs-platform-wallet/src/wallet/identity/network/private_contact_requests.rs @@ -0,0 +1,8551 @@ +//! DashPay private contact requests (protocol version 15): send, sync, accept. +//! +//! A private contact request is an entry of DashPay's `privateContactRequest` +//! store, sealed to one of the recipient's DashPay keys and appended by a +//! transition paid from the shielded pool, so nothing on chain names either +//! side. The sender names itself and its encryption key inside, signs it with +//! the identity signer it signs transitions with, and seals with the outgoing +//! viewing key of the encryption key, so a wallet restored from seed recovers +//! what it sent. +//! +//! The wallet keeps a private request as it keeps a public one, a +//! [`ContactRequest`], so contacts establish, rotate and get their DashPay +//! accounts the same way whichever way a request came. The entry carries the +//! compact xpub and the label in the clear; the wallet encrypts them under the +//! pair's DIP-15 shared key, as a `contactRequest` document carries them, with +//! IVs from the entry's commitment so a rescan rebuilds the same record, and at +//! core height 0, which no `contactRequest` document has: that a request came +//! privately is the record's own ([`ContactRequest::came_privately`]), so it +//! holds across restarts. The contact's shielded address, which a +//! `ContactRequest` has no field for, is kept in memory and read with +//! [`PrivateContact`]. +//! +//! The store is opened and requests are sealed inside the +//! [`ContactCryptoProvider`], which keeps the key material; requests are signed +//! by the identity signer. Checking an entry needs no secret key. +//! +//! [`PrivateContact`]: crate::wallet::identity::PrivateContact + +use std::collections::{BTreeMap, BTreeSet}; +use std::future::Future; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; + +use dash_sdk::dapi_grpc::platform::v0::ResponseMetadata; +use dash_sdk::platform::dashpay::{ + private_contact_request_identities, private_contact_request_signature, + private_contact_request_signing_data, private_contact_request_signing_keys, + private_contact_requests_from_scan, recipient_key_purpose_is_valid, PrivateContactRequest, + ProvedIdentity, RejectedPrivateContactRequest, PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, +}; +use dash_sdk::platform::private_document_entries::{PrivateDocumentEntries, PrivateDocumentEntry}; +use dash_sdk::platform::private_document_scan::{ + PrivateDocumentScanKeys, MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES, +}; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::decode_private_document_payload; +use dpp::identity::accessors::IdentityGettersV0; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::identity_public_key::Purpose; +use dpp::identity::signer::Signer; +use dpp::identity::{Identity, IdentityPublicKey, KeyID, KeyType}; +use dpp::platform_value::BinaryData; +use dpp::prelude::Identifier; +use dpp::secp256k1::rand::rngs::StdRng; +use dpp::secp256k1::rand::{RngCore, SeedableRng}; +use dpp::secp256k1::{PublicKey, SecretKey}; +use futures::StreamExt; +use key_wallet::bip32::{DerivationPath, KeyDerivationType}; +use key_wallet_manager::WalletManager; +use platform_encryption::{encrypt_account_label, encrypt_extended_public_key, COMPACT_XPUB_LEN}; +use tokio::sync::RwLock; +use zeroize::Zeroizing; + +use super::contact_requests::{ + ingest_received_requests, ingest_sent_requests, newest_received_per_sender, + newest_sent_per_recipient, predates, records_or_supersedes, tear_down_stale_external_accounts, + AutoAcceptProofSource, ContactCryptoProvider, PreparedContactRequest, ReplyOutcome, +}; +use super::signing_key::first_available; +use super::{dashpay_contract, DashPayView, IdentityWallet}; +use crate::broadcaster::TransactionBroadcaster; +use crate::changeset::{ContactChangeSet, PersistenceError, PrivateContactRequestsPosition}; +use crate::error::PlatformWalletError; +use crate::wallet::core::WalletGeneration; +use crate::wallet::identity::types::dashpay::contact_request::ContactRequest; +use crate::wallet::identity::types::dashpay::established_contact::EstablishedContact; +use crate::wallet::identity::ManagedIdentity; +use crate::wallet::persister::WalletPersister; +use crate::wallet::platform_wallet::{PlatformWalletInfo, WalletId}; + +/// The paths of the keys a wallet's identities read DashPay's private contact +/// request store with, for a [`ContactCryptoProvider`] to derive inside the +/// signer. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct PrivateDocumentKeyPaths { + /// Each identity entries may be sealed to, with the path of each of its + /// keys an entry may be sealed to, by key id. + pub recipients: Vec<(Identifier, Vec<(KeyID, DerivationPath)>)>, + /// Each identity that may have sealed entries, with the id and path of + /// each of its encryption keys: one outgoing viewing key each. + pub senders: Vec<(Identifier, KeyID, DerivationPath)>, +} + +impl PrivateDocumentKeyPaths { + /// The scan keys these paths derive to, `derive` deriving the secret key at + /// a path: for a provider implementation to build inside the signer and + /// drop there. + pub fn scan_keys( + &self, + derive: impl FnMut(&DerivationPath) -> Result, + ) -> Result { + PrivateDocumentScanKeys::derive(&self.recipients, &self.senders, derive) + } +} + +/// What a contact request from one of the wallet's identities waits for, as +/// the wallet decides it ([`DashPayView::private_contact_readiness`]), so a +/// host shows it rather than inferring it from the refusals it got. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct PrivateContactReadiness { + /// Whether the reservations of the wallet's private payments from an + /// earlier process are restored for the identity. While false, its public + /// contact requests, sent or accepted, are refused with + /// [`PlatformWalletError::PrivateContactPaymentsNotRestored`]: the wallet + /// bound a shielded account before it was loaded, or the identity was + /// loaded marking a private payment that may be unresolved. Configuring + /// the shielded store, or binding the wallet's shielded account, restores + /// them; no private sync does. A build without the `shielded` feature + /// never does. + pub private_payments_restored: bool, + /// Whether a private sync read DashPay's private contact request store to + /// its end for the identity since load. Until then its first send or + /// accept reads it first, and is refused with + /// [`PlatformWalletError::PrivateContactRequestPending`] while that read + /// cannot finish; a private sync sets it. + pub private_store_read: bool, + /// Whether a private contact request the identity paid for may still be + /// unresolved: the marker its position record stores + /// (`PrivateContactRequestsPosition::private_payments_unresolved`), set + /// while it holds a private request it is paying for or has not read back. + /// A contact's own pending request is + /// [`DashPayState::private_reply_pending`](crate::wallet::identity::DashPayState::private_reply_pending). + pub unresolved_private_payment: bool, +} + +/// What a pass over DashPay's private contact request store did. +#[derive(Clone, Debug, Default)] +pub struct PrivateContactSyncReport { + /// The store pages read. + pub pages_read: usize, + /// The requests this pass recorded, received and sent, as the wallet + /// records them: new ones and rotations. Not one already tracked, an + /// ignored sender's, a replay older than the tracked one, or one whose + /// store failed. + pub requests: Vec, + /// Entries that opened for one of the wallet's identities but were + /// refused. + pub rejected: Vec, + /// The store position the pass stopped at. + pub next_position: u64, + /// Whether the pass read to the store's last entry. + pub complete: bool, + /// Identities whose ingest did not reach disk, or one of whose sent + /// requests could not be rebuilt for want of its recipient: their cursor + /// stays where it was, so the next pass reads their entries again. + pub unpersisted_identities: Vec, +} + +/// How many pages of the store a send reads back after paying: one window of +/// pages fetched at once, enough for an identity that synced recently, whose +/// new entry is in the last page or two. A request not read back within them +/// stays pending for a later sync. +const POST_SEND_SYNC_PAGES: usize = MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES; + +/// How many pages of the store each private sync reads that a private send +/// runs, before it seals anything, when the store was not yet read to its end +/// since load: sixteen windows of pages fetched at once. With the read +/// position kept across restarts, the read is from there to the end. +const PRIVATE_STORE_READ_PAGES: usize = 16 * MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES; + +/// How many fetched pages a sync scans at once, one key derivation for them +/// all: at most the pages in flight. +const PRIVATE_SCAN_BATCH_PAGES: usize = MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES; + +/// How many such syncs a private send runs before it refuses with +/// [`PlatformWalletError::PrivateContactRequestPending`]: a later send goes on +/// from where they stopped. +const PRIVATE_STORE_READ_PASSES: usize = 4; + +/// One of the wallet's identities, as a pass reads the store for it. +struct ScannedIdentity { + identity_id: Identifier, + identity_index: u32, + /// Its cursor when the pass last advanced it; a change under the pass is a + /// rewind, which the pass leaves alone. + position: u64, + /// Its ingest failed this pass: it is not advanced again until the next. + failed: bool, + /// Its cursor was rewound under the pass (unignoring a sender, or its + /// scanning keys changing): it is not advanced again this pass, so the + /// next reads from the rewound position. + rewound: bool, + /// The fingerprint of the keys the pass scans with for it + /// (`ManagedIdentity::private_scan_key_fingerprint`): progress made + /// with them holds only while they are still the identity's. + key_fingerprint: [u8; 32], +} + +impl ScannedIdentity { + /// Advance `managed`'s cursor to `next_position` once a page's ingest for + /// it reached disk: unless it was rewound under the pass, which stops it + /// advancing for the rest of the pass, as a later page's advance would + /// otherwise overwrite the rewind. + fn advance(&mut self, managed: &mut ManagedIdentity, next_position: u64) { + if self.failed || self.rewound { + return; + } + // Keys added under the pass opened nothing it read. + if !self.keys_current(managed) { + self.rewound = true; + return; + } + if managed.dashpay().private_contact_requests_position() != self.position { + self.rewound = true; + return; + } + managed.advance_private_contact_requests_position(self.position, next_position); + self.position = managed.dashpay().private_contact_requests_position(); + } + + /// Whether the keys the pass scans with are still the identity's, and + /// the ones its read position holds for. + fn keys_current(&self, managed: &ManagedIdentity) -> bool { + managed.private_scan_key_fingerprint() == self.key_fingerprint + && managed + .private_contact_requests_position_record() + .key_fingerprint + == self.key_fingerprint + } +} + +/// Handed to a private request's `pay`, which marks it right before anything +/// it does may reach the network. Until it is marked, a send dropped (a +/// cancelled accept, a timeout) releases its reservation, as nothing was paid +/// and no sync could settle it; once marked, the reservation stays for a +/// sync to settle. A `pay` that awaits anything first, such as resolving its +/// payment source, a spend key or a proof, marks it only after those. +/// +/// It also names the request it pays for ([`Self::account_reference`]), so +/// the payment's record can be saved with it and a restart restores the +/// reservation without opening the entry. +/// +/// A payment that marked it and then failed keeps the reservation too: its +/// transition may still execute. Only one whose transition provably never +/// left the wallet ([`Self::note_never_delivered`]) releases it. +#[derive(Clone, Debug, Default)] +pub struct PaymentStarted { + marked: Arc, + never_delivered: Arc, + account_reference: Option, +} + +impl PaymentStarted { + /// The payment may reach the network from now on. + pub fn mark(&self) { + self.marked.store(true, Ordering::Release); + } + + /// Whether [`Self::mark`] was called. + pub fn is_marked(&self) -> bool { + self.marked.load(Ordering::Acquire) + } + + /// The payment's transition provably never left the wallet, and the + /// shielded store's journal record of it is discharged: nothing can + /// execute it, so its reservation may be released. + #[cfg(feature = "shielded")] + pub(crate) fn note_never_delivered(&self) { + self.never_delivered.store(true, Ordering::Release); + } + + /// Whether the payment may have reached the network: it was marked + /// started, and its transition was not shown never to have left. + pub(crate) fn may_have_reached_network(&self) -> bool { + self.is_marked() && !self.never_delivered.load(Ordering::Acquire) + } + + /// The account reference of the private request this payment is for, + /// when the send handed it over. + pub fn account_reference(&self) -> Option { + self.account_reference + } + + /// The same token, naming the request with `account_reference`. + fn for_request(&self, account_reference: u32) -> Self { + Self { + marked: Arc::clone(&self.marked), + never_delivered: Arc::clone(&self.never_delivered), + account_reference: Some(account_reference), + } + } +} + +/// The reservation one send attempt holds on the private request to a +/// recipient that may be in flight. Dropped before its payment is marked +/// started ([`PaymentStarted`]), it releases the reservation: a send +/// cancelled before it pays leaves nothing pending. Once the payment may +/// have started, or the send kept it, the reservation stays for the sync. +struct PrivateReplyReservation { + wallet_manager: Arc>>, + persister: WalletPersister, + wallet_id: WalletId, + sender: Identifier, + recipient: Identifier, + attempt: u64, + payment_started: PaymentStarted, + /// Whether the payment may have reached the network when it returned: + /// marked by the payment itself and not shown never to have left + /// ([`PaymentStarted::may_have_reached_network`]). + payment_may_have_executed: bool, + kept: bool, +} + +impl PrivateReplyReservation { + /// Leave the reservation for the private sync to settle. + fn keep(&mut self) { + self.kept = true; + } + + /// Release the reservation now, kept or not. Dropped while it waits for + /// the wallet lock, the guard releases it. + async fn release(mut self) { + self.kept = false; + self.payment_started = PaymentStarted::default(); + let mut wm = self.wallet_manager.write().await; + release_reservation( + &mut wm, + &self.wallet_id, + &self.sender, + &self.recipient, + self.attempt, + ); + store_private_payment_marker(&self.persister, &wm, &self.wallet_id, &self.sender); + self.kept = true; + } +} + +impl Drop for PrivateReplyReservation { + fn drop(&mut self) { + if self.kept || self.payment_started.is_marked() { + return; + } + let (wallet_id, sender, recipient, attempt) = + (self.wallet_id, self.sender, self.recipient, self.attempt); + let persister = self.persister.clone(); + apply_detached( + &self.wallet_manager, + "a cancelled private contact request's reservation", + move |wm| { + release_reservation(wm, &wallet_id, &sender, &recipient, attempt); + store_private_payment_marker(&persister, wm, &wallet_id, &sender); + }, + ); + } +} + +/// Run `apply` on the wallet manager, for a guard released on drop, which +/// cannot wait for the lock: now when the lock is free, otherwise on a +/// spawned task. Outside a runtime it is logged, and `what` stays until +/// restart. +pub(super) fn apply_detached( + wallet_manager: &Arc>>, + what: &'static str, + apply: impl FnOnce(&mut WalletManager) + Send + 'static, +) { + if let Ok(mut wm) = wallet_manager.try_write() { + apply(&mut wm); + return; + } + let Ok(runtime) = tokio::runtime::Handle::try_current() else { + tracing::warn!( + what, + "released on drop outside a runtime; it stays until restart" + ); + return; + }; + let wallet_manager = Arc::clone(wallet_manager); + runtime.spawn(async move { + let mut wm = wallet_manager.write().await; + apply(&mut wm); + }); +} + +/// Release the reservation `attempt` holds on `sender`'s private request to +/// `recipient`. +fn release_reservation( + wm: &mut WalletManager, + wallet_id: &WalletId, + sender: &Identifier, + recipient: &Identifier, + attempt: u64, +) { + if let Some(managed) = wm + .get_wallet_info_mut(wallet_id) + .and_then(|info| info.identity_manager.managed_identity_mut(sender)) + { + managed.release_private_reply(recipient, attempt); + } +} + +/// Note `read_back`, the commitments of private requests a private sync read +/// back and recorded, for their journal records to be discharged +/// (`WalletPersister::note_read_back_private_entries`), once what records +/// them is durable: at once for a backend that commits inline, after a flush +/// otherwise. A failed flush notes nothing, which keeps the records, and so +/// the pairs held, until a later read. +fn note_read_back_once_durable(persister: &WalletPersister, read_back: Vec<[u8; 32]>) { + if !persister.store_commits_inline() { + if let Err(error) = persister.flush() { + tracing::warn!( + %error, + "private requests read back are not durably recorded; their journal records \ + stay" + ); + return; + } + } + persister.note_read_back_private_entries(read_back); +} + +/// A private contact request payment a restoration reads, from the shielded +/// store's journal or from the redrive record of its spend: (sender, +/// recipient, account reference, the entry's commitment). +pub(crate) type JournaledPrivatePayment = (Identifier, Identifier, u32, [u8; 32]); + +/// What [`reserve_unresolved_private_payments`] reserved. +#[derive(Debug, Default)] +pub(crate) struct ReservedPrivatePayments { + /// The (sender, recipient) pairs reserved, each once, in the payments' + /// order. + pub(crate) restored: Vec<(Identifier, Identifier)>, + /// The pairs of payments whose sender the wallet does not manage. + #[cfg_attr(not(feature = "shielded"), allow(dead_code))] + pub(crate) absent: BTreeSet<(Identifier, Identifier)>, +} + +/// The one step every restoration of a private contact request payment's +/// reservation takes, and every retarget of one: reserve the recipient of each +/// of `payments` that may still execute, in `info`, which the caller holds +/// under the wallet manager's write lock. +/// +/// Fenced to `generation`, the wallet generation the caller's handle belongs +/// to: a wallet removed, or re-created under its id, since then is refused +/// with [`PlatformWalletError::WalletNotFound`] and nothing is reserved. The +/// check and the reservations are under the one lock a removal takes. +/// +/// Left alone: +/// - a payment whose journal record was discharged in this session +/// ([`WalletPersister::private_entry_discharged`]): a restoration that read +/// it before the discharge, and reserves after the reservation was released, +/// would hold a pair no entry ever settles; +/// - one whose request the sender's tracked request to its recipient records +/// or supersedes, as a private sync that recorded it settled it; +/// - one whose sender the wallet does not manage, returned in +/// [`ReservedPrivatePayments::absent`]. +/// +/// The pair `paying` is retargeted to the entry of its first payment left to +/// reserve, as the entry about to be paid is not paid for; every other pair +/// keeps the entry it is reserved for, and gets one only when it has none. +pub(crate) fn reserve_unresolved_private_payments( + info: &mut PlatformWalletInfo, + generation: &Arc, + wallet_id: WalletId, + persister: &WalletPersister, + payments: &[JournaledPrivatePayment], + paying: Option<(Identifier, Identifier)>, +) -> Result { + if !Arc::ptr_eq(&info.generation, generation) { + return Err(PlatformWalletError::WalletNotFound(hex::encode(wallet_id))); + } + let mut reserved = ReservedPrivatePayments::default(); + for &(sender, recipient, account_reference, commitment) in payments { + if persister.private_entry_discharged(&commitment) { + continue; + } + let Some(managed) = info.identity_manager.managed_identity_mut(&sender) else { + reserved.absent.insert((sender, recipient)); + continue; + }; + if managed + .dashpay() + .outgoing_request(&recipient) + .is_some_and(|tracked| records_or_supersedes(tracked, account_reference)) + { + continue; + } + let first = !reserved.restored.contains(&(sender, recipient)); + if paying == Some((sender, recipient)) { + if first { + managed.retarget_private_reply(recipient, commitment); + } + } else { + managed.restore_private_reply(recipient, commitment); + } + if first { + reserved.restored.push((sender, recipient)); + } + } + Ok(reserved) +} + +/// The body of [`DashPayView::restore_private_payments`], over the wallet's +/// state alone, so a restoration deferred to a task while the shielded store +/// is busy runs it too. +#[cfg(feature = "shielded")] +pub(crate) async fn restore_journaled_private_payments( + wallet_manager: &Arc>>, + generation: &Arc, + persister: &WalletPersister, + wallet_id: WalletId, + payments: &[JournaledPrivatePayment], + journal_authoritative: bool, +) -> Result, PlatformWalletError> { + let mut wm = wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(wallet_id)))?; + let ReservedPrivatePayments { restored, absent } = reserve_unresolved_private_payments( + info, generation, wallet_id, persister, payments, None, + )?; + persister.hold_absent_sender_private_pairs(absent); + // Under the same lock, after the reservations: a send that sees an + // identity's hold gone sees them. + let identity_ids: Vec = info + .identity_manager + .all_identities() + .iter() + .map(|identity| identity.id()) + .collect(); + let mut unaccounted = 0usize; + for identity_id in identity_ids { + let Some(managed) = info.identity_manager.managed_identity_mut(&identity_id) else { + continue; + }; + if !managed.dashpay().private_payments_unrestored() { + continue; + } + if !journal_authoritative && !managed.dashpay().private_replies_pending() { + unaccounted += 1; + continue; + } + if managed.note_private_payments_restored() + && persister.persists_private_contact_scan_state() + { + store_private_position( + persister, + identity_id, + managed.private_contact_requests_position_record(), + ); + } + } + if unaccounted > 0 { + return Err(PlatformWalletError::ShieldedRecoveryCorrupted { + account_index: None, + reason: format!( + "{unaccounted} of the wallet's identities were loaded marking a private contact \ + request payment that may be unresolved, and the shielded store's private \ + payment journal for the wallet is new, so it cannot show that payment \ + resolved: their contact requests stay held" + ), + }); + } + Ok(restored) +} + +/// Store `owner`'s private read position record again, for a backend that +/// keeps it, as whether a private payment of `owner` may be unresolved +/// (`PrivateContactRequestsPosition::private_payments_unresolved`) may have +/// changed: a reservation was released or settled. Flushed when the backend +/// does not commit a store inline, so a cleared marker does not hold the +/// identity after a restart. A store or flush that fails is logged and +/// leaves the marker set, which holds the identity after a restart only +/// until the next restoration: that one clears it, the shielded store's +/// journal being open by then and showing the payment resolved. Never the +/// other way round. +fn store_private_payment_marker( + persister: &WalletPersister, + wm: &WalletManager, + wallet_id: &WalletId, + owner: &Identifier, +) { + if !persister.persists_private_contact_scan_state() { + return; + } + if let Some(managed) = wm + .get_wallet_info(wallet_id) + .and_then(|info| info.identity_manager.managed_identity(owner)) + { + store_private_position( + persister, + *owner, + managed.private_contact_requests_position_record(), + ); + if !persister.store_commits_inline() { + if let Err(error) = persister.flush() { + tracing::warn!( + owner = %owner, + %error, + "a private payment marker change is not flushed; the next restoration \ + clears a stale one" + ); + } + } + } +} + +/// The changeset recording, for each contact one of `addresses` came from, the +/// shielded address `owner` keeps for it now, or its removal. +fn shielded_address_changeset( + managed: &ManagedIdentity, + owner: Identifier, + addresses: &[(ContactRequest, Option<[u8; 43]>)], +) -> ContactChangeSet { + let mut changeset = ContactChangeSet::default(); + for (record, _) in addresses { + let key = (owner, record.sender_id); + match managed + .dashpay() + .private_shielded_address(&record.sender_id) + { + Some(address) => { + changeset.private_shielded_addresses.insert(key, address); + } + None => { + changeset.removed_private_shielded_addresses.insert(key); + } + } + } + changeset +} + +/// Store `owner`'s private read position. A failed store leaves the cursor +/// advanced in memory: the next launch reads from the last position stored, +/// and ingest is a fixpoint. +fn store_private_position( + persister: &WalletPersister, + owner: Identifier, + position: PrivateContactRequestsPosition, +) { + let mut changeset = ContactChangeSet::default(); + changeset + .private_contact_requests_positions + .insert(owner, position); + if let Err(error) = persister.store(changeset.into()) { + tracing::warn!( + owner = %owner, position = position.position, %error, + "private read position not stored; the next launch reads from the last one stored" + ); + } +} + +/// The local record of a private request: a [`ContactRequest`] carrying the +/// request's compact xpub and label encrypted under `shared_key`, the pair's +/// DIP-15 shared key, as a `contactRequest` document carries them. The IVs are +/// the two halves of the entry's `commitment`, so a rescan of the entry +/// rebuilds the same record, and the wallet's comparisons of a contact's +/// encrypted xpub hold across passes. Its core height is 0, which marks it +/// private ([`ContactRequest::came_privately`]); our receiving account for the +/// contact is then scanned from the wallet's birth. +#[allow(clippy::too_many_arguments)] +pub(super) fn private_request_record( + sender_id: Identifier, + recipient_id: Identifier, + sender_key_index: KeyID, + recipient_key_index: KeyID, + request: &PrivateContactRequest, + commitment: &[u8; 32], + shared_key: &[u8; 32], + created_at: u64, +) -> ContactRequest { + let mut xpub_iv = [0u8; 16]; + let mut label_iv = [0u8; 16]; + xpub_iv.copy_from_slice(&commitment[..16]); + label_iv.copy_from_slice(&commitment[16..]); + let mut record = ContactRequest::new( + sender_id, + recipient_id, + sender_key_index, + recipient_key_index, + request.account_reference, + encrypt_extended_public_key(shared_key, &xpub_iv, &request.extended_public_key), + 0, + created_at, + ); + record.encrypted_account_label = request + .account_label + .as_deref() + .map(|label| encrypt_account_label(shared_key, &label_iv, label)); + record.auto_accept_proof = request.auto_accept_proof.clone(); + record +} + +/// The public key of `identity`'s key `key_id`. +fn identity_public_key( + identity: &Identity, + key_id: KeyID, +) -> Result { + let key = identity.public_keys().get(&key_id).ok_or_else(|| { + PlatformWalletError::InvalidIdentityData(format!( + "identity {} has no key {key_id}", + identity.id() + )) + })?; + PublicKey::from_slice(key.data().as_slice()).map_err(|e| { + PlatformWalletError::InvalidIdentityData(format!( + "key {key_id} of identity {} is not a secp256k1 key: {e}", + identity.id() + )) + }) +} + +impl DashPayView<'_, B> { + /// Send a private contact request from `sender_identity_id` to + /// `recipient_identity_id`: the request a `contactRequest` document would + /// carry (our receiving xpub for the pair, the masked account reference, + /// the label and the auto-accept proof) plus our `shielded_address`, + /// signed by `signer` with one of the sender's signing keys, sealed in the + /// crypto provider to the recipient's DashPay key, then appended by `pay`, + /// which pays for it from the shielded pool and returns the metadata of + /// the proof that it was appended + /// (`PlatformWallet::send_private_contact_request`). + /// + /// The keys and the account reference are chosen as for a document send, + /// so a later request to the same recipient, either way, rotates past this + /// one. + /// + /// One private request to a recipient is in flight at a time: it is + /// reserved before sealing, and while it is pending another is refused + /// with [`PlatformWalletError::PrivateContactRequestPending`], as the store + /// keeps every entry appended and a second payment could execute too. A + /// public request to the recipient is refused the same way. + /// + /// The proof `pay` waits for shows the notes spent, not the entry + /// appended, so a paid request is recorded only once read back: a private + /// sync of a few pages follows the payment, and records the request from + /// the store as it records any request the identity sent. If it is not + /// read back yet, the send returns `PrivateContactRequestPending` and the + /// request stays pending until a later sync reads it. An unconfirmed + /// payment stays pending the same way; one that definitely failed + /// releases it, and so does a send dropped before its payment started: + /// `pay` gets a [`PaymentStarted`] it marks right before anything it does + /// may reach the network, after any resolving it awaits first. `pay` itself + /// may return `PrivateContactRequestPending` to keep the request pending + /// unpaid. The pending request is kept in memory only; after a restart, + /// [`Self::restore_pending_private_contact_requests`] restores those whose + /// payment is unconfirmed. + /// + /// A request paid for and confirmed but not read back leaves nothing + /// that survives a restart. So the first private send or accept from an + /// identity since load reads the store to its end first, which records + /// such a request; while that read cannot finish, the send is refused + /// with `PrivateContactRequestPending`, nothing sealed or paid. + #[allow(clippy::too_many_arguments)] + pub async fn send_private_contact_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + pay: F, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + self.send_private_request( + sender_identity_id, + recipient_identity_id, + account_label, + auto_accept_proof, + shielded_address, + signer, + crypto, + false, + pay, + ) + .await? + .ok_or_else(|| { + PlatformWalletError::InvalidParameter( + "no private contact request was sent: only a reply adopts one already sent" + .to_string(), + ) + }) + } + + /// Reply privately to `request`, paid by `pay`: `AlreadyReplied`, with + /// nothing sealed or paid, when a request to its sender is tracked by the + /// time the pair is reserved (another accept's reply, recorded in + /// between), which the accept then adopts. + pub(super) async fn send_private_reply( + &self, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + pay: F, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + Ok( + match self + .send_private_request( + &request.recipient_id, + &request.sender_id, + None, + AutoAcceptProofSource::None, + shielded_address, + signer, + crypto, + true, + pay, + ) + .await? + { + Some(_) => ReplyOutcome::Sent, + None => ReplyOutcome::AlreadyReplied, + }, + ) + } + + /// [`Self::send_private_contact_request`], for a reply when + /// `adopt_reply`: then a request to the recipient tracked once the store + /// was read and the pair is reserved, in the same lock, is adopted + /// rather than rotated past, and `None` comes back with nothing paid. + #[allow(clippy::too_many_arguments)] + async fn send_private_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + adopt_reply: bool, + pay: F, + ) -> Result, PlatformWalletError> + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + // Before anything is read, sealed or paid: a payment no restart could + // see would leave its pair unheld. + if !self.persister.persists_private_contact_scan_state() { + return Err(PlatformWalletError::PrivateContactRecoveryUnsupported); + } + // A private payment from an earlier process may hold this pair, which + // only a restoration from the shielded store's journal shows: until + // one succeeds, no private request is paid for either. + if self.persister.private_payments_unrestored() + || self.private_payments_unrestored(sender_identity_id).await? + { + return Err(PlatformWalletError::PrivateContactPaymentsNotRestored); + } + self.read_private_store_before_sending(sender_identity_id, recipient_identity_id, crypto) + .await?; + self.release_discharged_private_requests().await?; + let absent_sender_held = self + .persister + .absent_sender_private_pair_held(sender_identity_id, recipient_identity_id); + let reserved = self + .with_managed(sender_identity_id, |managed| { + if adopt_reply + && managed + .dashpay() + .outgoing_request(recipient_identity_id) + .is_some() + { + return None; + } + if absent_sender_held { + return Some(None); + } + Some(managed.reserve_private_reply(*recipient_identity_id)) + }) + .await?; + let Some(reserved) = reserved else { + return Ok(None); + }; + let attempt = reserved.ok_or(PlatformWalletError::PrivateContactRequestPending( + *recipient_identity_id, + ))?; + let mut reservation = PrivateReplyReservation { + wallet_manager: Arc::clone(&self.wallet_manager), + persister: self.persister.clone(), + wallet_id: self.wallet_id, + sender: *sender_identity_id, + recipient: *recipient_identity_id, + attempt, + payment_started: PaymentStarted::default(), + payment_may_have_executed: false, + kept: false, + }; + let sent = self + .send_reserved_private_contact_request( + sender_identity_id, + recipient_identity_id, + account_label, + auto_accept_proof, + shielded_address, + signer, + crypto, + &mut reservation, + pay, + ) + .await; + // A payment that may have executed keeps the request pending, its + // marker set: one that reached the network and then failed too, as + // its entry may still be appended, unless its transition provably + // never left the wallet. + let keep_pending = reservation.payment_may_have_executed + || matches!( + &sent, + Err(PlatformWalletError::PrivateContactRequestPending(_) + | PlatformWalletError::ShieldedSpendUnconfirmed { .. } + | PlatformWalletError::ShieldedBroadcastMaybeDelivered(_)) + ); + if keep_pending { + reservation.keep(); + } else { + reservation.release().await; + } + sent.map(Some) + } + + /// Make sure DashPay's private contact request store was read to its end + /// for `identity_id` since load, before a private request from it to + /// `recipient` is sealed or paid for: a request paid for in an earlier + /// process may be in the store unread, and the reservation that kept a + /// second one from being paid did not survive the restart. Until it was, + /// this runs private syncs of [`PRIVATE_STORE_READ_PAGES`] pages, at most + /// [`PRIVATE_STORE_READ_PASSES`] of them, and refuses with + /// [`PlatformWalletError::PrivateContactRequestPending`] when they stop + /// advancing or run out first. A sync error comes back as it is. Once + /// read, it reads nothing. An identity the private sync does not read, + /// with no HD index or no key an entry can be sealed to or from, sends no + /// private request, so it is not held. + async fn read_private_store_before_sending( + &self, + identity_id: &Identifier, + recipient: &Identifier, + crypto: &C, + ) -> Result<(), PlatformWalletError> + where + C: ContactCryptoProvider + Sync, + { + // Keys added since the store was read clear the flag first. + self.reconcile_private_scan_keys().await?; + if self.private_store_read(identity_id).await? { + return Ok(()); + } + let (identities, _) = self.private_scan_identities().await?; + if !identities + .iter() + .any(|identity| identity.identity_id == *identity_id) + { + return Ok(()); + } + let mut position = identities + .iter() + .map(|identity| identity.position) + .min() + .unwrap_or_default(); + for _ in 0..PRIVATE_STORE_READ_PASSES { + let report = self + .sync_private_contact_requests(crypto, PRIVATE_STORE_READ_PAGES) + .await?; + if self.private_store_read(identity_id).await? { + return Ok(()); + } + if report.next_position <= position { + break; + } + position = report.next_position; + } + Err(PlatformWalletError::PrivateContactRequestPending( + *recipient, + )) + } + + /// [`Self::read_private_store_before_sending`], before a public request + /// or reply from `identity_id` to `recipient` is chosen or sent, so a + /// private request paid for in an earlier process holds the pair here + /// too. A network that serves no private documents yet holds none, so + /// the SDK's refusal of the read reads as nothing to read, but only once + /// a verified answer shows the network below the protocol version that + /// serves them (`Sdk::network_proven_below`): a refusal alone does not + /// say whether the network or this SDK is below it, and an SDK pinned + /// below it refuses a store the network has. Without that evidence the + /// refusal comes back as it is, and nothing is sent. + /// + /// Refused first, with nothing read, while the wallet's unconfirmed + /// private payments from an earlier process may be unrestored + /// ([`PlatformWalletError::PrivateContactPaymentsNotRestored`]): such a + /// payment's entry is not in the store until it executes, so no read + /// shows the pair it holds; only the shielded store's record of it does. + /// That is so for a wallet a shielded build loads with a persisted + /// viewing key, and, in every build, for an identity loaded with its + /// position record marking a private payment that may be unresolved + /// (`PrivateContactRequestsPosition::private_payments_unresolved`). + pub(super) async fn read_private_store_before_deciding( + &self, + identity_id: &Identifier, + recipient: &Identifier, + crypto: &C, + ) -> Result<(), PlatformWalletError> + where + C: ContactCryptoProvider + Sync, + { + if self.persister.private_payments_unrestored() + || self.private_payments_unrestored(identity_id).await? + { + return Err(PlatformWalletError::PrivateContactPaymentsNotRestored); + } + match self + .read_private_store_before_sending(identity_id, recipient, crypto) + .await + { + Err(PlatformWalletError::Sdk( + refusal @ dash_sdk::Error::ProtocolVersionNotActive { required, .. }, + )) => match self.sdk.network_proven_below(required).await { + Ok(true) => Ok(()), + Ok(false) => Err(PlatformWalletError::Sdk(refusal)), + Err(error) => { + tracing::warn!( + %error, + "the network's protocol version could not be verified; the private \ + store is not taken as empty" + ); + Err(PlatformWalletError::Sdk(refusal)) + } + }, + read => read, + } + } + + /// Rewind the private read position and the read flag of each of the + /// wallet's identities whose scanning keys changed since they were set + /// (`ManagedIdentity::reconcile_private_scan_keys`), storing the rewind + /// for a backend that keeps the position. A refresh that folds in a key + /// another device added is caught here, before the next scan or send. + pub(crate) async fn reconcile_private_scan_keys(&self) -> Result<(), PlatformWalletError> { + let persists_scan_state = self.persister.persists_private_contact_scan_state(); + let mut wm = self.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + let identity_ids: Vec = info + .identity_manager + .all_identities() + .iter() + .map(|identity| identity.id()) + .collect(); + for identity_id in identity_ids { + let Some(managed) = info.identity_manager.managed_identity_mut(&identity_id) else { + continue; + }; + if let Some(position) = managed.reconcile_private_scan_keys() { + if persists_scan_state { + store_private_position(&self.persister, identity_id, position); + } + } + } + Ok(()) + } + + /// Store `owner`'s position record, marking it as holding a private payment + /// that may be unresolved, durably, before anything is paid: flushed when + /// the backend does not commit a store inline. A backend that does not + /// keep the marker refuses the payment + /// ([`PlatformWalletError::PrivateContactRecoveryUnsupported`]), and a + /// failed store or flush refuses it too, with nothing paid: a payment a + /// restart could not see would leave the pair unheld. + /// + /// Clearing the marker once the payment resolves is not flushed: a clear + /// that is lost only holds the identity after a restart until a + /// restoration clears it, which fails closed. + fn store_private_payment_marker_before_paying( + &self, + owner: &Identifier, + marker: PrivateContactRequestsPosition, + ) -> Result<(), PlatformWalletError> { + if !self.persister.persists_private_contact_scan_state() { + return Err(PlatformWalletError::PrivateContactRecoveryUnsupported); + } + let mut changeset = ContactChangeSet::default(); + changeset + .private_contact_requests_positions + .insert(*owner, marker); + let not_kept = |error: PersistenceError| { + PlatformWalletError::Persistence(format!( + "the private contact request's unresolved payment was not stored durably, so it \ + is not paid for: {error}" + )) + }; + self.persister.store(changeset.into()).map_err(not_kept)?; + if !self.persister.store_commits_inline() { + self.persister.flush().map_err(not_kept)?; + } + Ok(()) + } + + /// Whether `identity_id` was loaded with a private contact request payment + /// that may be unresolved and no restoration has restored it since + /// ([`DashPayState::private_payments_unrestored`]). + /// + /// [`DashPayState::private_payments_unrestored`]: crate::wallet::identity::DashPayState::private_payments_unrestored + pub(crate) async fn private_payments_unrestored( + &self, + identity_id: &Identifier, + ) -> Result { + let wm = self.wallet_manager.read().await; + let managed = wm + .get_wallet_info(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))? + .identity_manager + .managed_identity(identity_id) + .ok_or(PlatformWalletError::IdentityNotFound(*identity_id))?; + Ok(managed.dashpay().private_payments_unrestored()) + } + + /// What a contact request from `identity_id` waits for, as the wallet + /// decides it ([`PrivateContactReadiness`]). A pure read. + pub async fn private_contact_readiness( + &self, + identity_id: &Identifier, + ) -> Result { + let wallet_held = self.persister.private_payments_unrestored(); + let wm = self.wallet_manager.read().await; + let dashpay = wm + .get_wallet_info(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))? + .identity_manager + .managed_identity(identity_id) + .ok_or(PlatformWalletError::IdentityNotFound(*identity_id))? + .dashpay(); + Ok(PrivateContactReadiness { + private_payments_restored: !wallet_held && !dashpay.private_payments_unrestored(), + private_store_read: dashpay.private_contact_requests_read(), + unresolved_private_payment: dashpay.private_payments_unresolved(), + }) + } + + /// Whether any of the wallet's identities was loaded with a private + /// contact request payment that may be unresolved and no restoration has + /// restored it since. + pub(crate) async fn any_private_payments_unrestored(&self) -> bool { + let wm = self.wallet_manager.read().await; + wm.get_wallet_info(&self.wallet_id).is_some_and(|info| { + info.identity_manager + .all_identities() + .iter() + .filter_map(|identity| info.identity_manager.managed_identity(&identity.id())) + .any(|managed| managed.dashpay().private_payments_unrestored()) + }) + } + + /// Release the pending private requests whose payment's journal record + /// was discharged (`WalletPersister::note_discharged_private_entries`): + /// one proven expired, its anchor pruned with an input still unspent, can + /// never be read back. Run before a private request is reserved, before a + /// public one claims its pair, and by the private sync, so a retry needs + /// no restart. An identity's marker is stored again when it changed. + pub(crate) async fn release_discharged_private_requests( + &self, + ) -> Result<(), PlatformWalletError> { + if !self.persister.has_unreleased_discharged_private_entries() { + return Ok(()); + } + // Taken only once the wallet is locked and found, and applied with + // nothing awaited in between. + let mut wm = self.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + let discharged = self.persister.take_unreleased_discharged_private_entries(); + let identity_ids: Vec = info + .identity_manager + .all_identities() + .iter() + .map(|identity| identity.id()) + .collect(); + let mut changed = Vec::new(); + for identity_id in identity_ids { + if let Some(managed) = info.identity_manager.managed_identity_mut(&identity_id) { + let unresolved = managed.dashpay().private_payments_unresolved(); + managed.release_discharged_private_replies(&discharged); + if managed.dashpay().private_payments_unresolved() != unresolved { + changed.push(identity_id); + } + } + } + for identity_id in changed { + store_private_payment_marker(&self.persister, &wm, &self.wallet_id, &identity_id); + } + Ok(()) + } + + /// Whether a private sync read DashPay's private contact request store + /// to its end for `identity_id` since load. + async fn private_store_read( + &self, + identity_id: &Identifier, + ) -> Result { + let wm = self.wallet_manager.read().await; + let managed = wm + .get_wallet_info(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))? + .identity_manager + .managed_identity(identity_id) + .ok_or(PlatformWalletError::IdentityNotFound(*identity_id))?; + Ok(managed.dashpay().private_contact_requests_read()) + } + + /// Run `f` on the managed identity `identity_id` under the wallet lock. + async fn with_managed( + &self, + identity_id: &Identifier, + f: impl FnOnce(&mut ManagedIdentity) -> R, + ) -> Result { + let mut wm = self.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))? + .identity_manager + .managed_identity_mut(identity_id) + .ok_or(PlatformWalletError::IdentityNotFound(*identity_id))?; + Ok(f(managed)) + } + + /// The body of [`Self::send_private_contact_request`], once the request + /// to the recipient is reserved by `reservation`, which it keeps right + /// before paying. + #[allow(clippy::too_many_arguments)] + async fn send_reserved_private_contact_request( + &self, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + reservation: &mut PrivateReplyReservation, + pay: F, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + let PreparedContactRequest { + sender_identity, + sender_key_index, + sender_enc_path, + recipient_identity, + recipient_key_index, + recipient_enc_pubkey, + account_index, + contact_xpub_ext, + xpub_bytes, + account_reference, + auto_accept_proof, + } = self + .prepare_contact_request( + sender_identity_id, + recipient_identity_id, + auto_accept_proof, + crypto, + ) + .await?; + let extended_public_key: [u8; COMPACT_XPUB_LEN] = + xpub_bytes.as_slice().try_into().map_err(|_| { + PlatformWalletError::InvalidIdentityData(format!( + "the receiving xpub is {} bytes, not the {COMPACT_XPUB_LEN}-byte compact form", + xpub_bytes.len() + )) + })?; + let request = PrivateContactRequest { + account_reference, + extended_public_key, + account_label, + auto_accept_proof, + shielded_address, + }; + + // The request is signed as a public request's transition is: by the + // first signing key of the sender the signer holds, of those the SDK + // lets sign it (level, type, contract bounds). + let contract = dashpay_contract()?; + let signing_key = first_available( + private_contact_request_signing_keys(&contract, &sender_identity)?.into_iter(), + signer, + )? + .cloned() + .ok_or_else(|| { + PlatformWalletError::InvalidIdentityData( + "the sender identity has no enabled AUTHENTICATION key at HIGH or CRITICAL, \ + bound to no other contract, to sign a private contact request with" + .to_string(), + ) + })?; + let signing_data = private_contact_request_signing_data( + &contract, + &sender_identity, + sender_key_index, + &recipient_identity, + recipient_key_index, + &request, + self.sdk.version(), + )?; + let signature = + private_contact_request_signature(signer, &signing_key, &signing_data).await?; + + // Wiped on drop, so a send dropped while the seal is awaited wipes it too. + let mut esk_seed = Zeroizing::new([0u8; 32]); + StdRng::try_from_os_rng() + .map_err(|error| { + PlatformWalletError::InvalidIdentityData(format!( + "the OS RNG failed while sealing a private contact request: {error}" + )) + })? + .fill_bytes(&mut *esk_seed); + let sealed = crypto + .seal_private_contact_request( + &sender_enc_path, + &contract, + &sender_identity, + sender_key_index, + signing_key.id(), + &signature, + &recipient_identity, + recipient_key_index, + &request, + &esk_seed, + self.sdk.version(), + ) + .await; + drop(esk_seed); + let entry = sealed?; + // An entry starts with its 32-byte commitment. + let commitment: [u8; 32] = entry + .as_slice() + .get(..32) + .and_then(|commitment| commitment.try_into().ok()) + .ok_or_else(|| { + PlatformWalletError::InvalidIdentityData( + "the provider sealed an entry shorter than its commitment".to_string(), + ) + })?; + let marker = self + .with_managed(sender_identity_id, |managed| { + managed.note_private_reply_commitment( + *recipient_identity_id, + reservation.attempt, + commitment, + ); + managed.private_contact_requests_position_record() + }) + .await?; + // Before anything is paid, the identity is marked as holding a private + // payment that may be unresolved: a process reopening the wallet before + // the payment's reservation is restored, or in a build that cannot + // restore it, holds its public requests until then. + self.store_private_payment_marker_before_paying(sender_identity_id, marker)?; + // What the record read back must carry, computed before paying. + let shared_secret = crypto + .ecdh_shared_secret(&sender_enc_path, &recipient_enc_pubkey) + .await?; + let expected = private_request_record( + *sender_identity_id, + *recipient_identity_id, + sender_key_index, + recipient_key_index, + &request, + &commitment, + &shared_secret, + 0, + ); + + // `pay` marks the payment started right before it may reach the + // network: from then on the reservation is the sync's to settle, + // whatever happens to this send. + let paid = pay( + entry, + reservation.payment_started.for_request(account_reference), + ) + .await; + reservation.payment_may_have_executed = + reservation.payment_started.may_have_reached_network(); + reservation.payment_started.mark(); + paid?; + + // Recorded only as the store proves it: read it back. + if let Err(error) = self + .sync_private_contact_requests(crypto, POST_SEND_SYNC_PAGES) + .await + { + tracing::warn!( + sender = %sender_identity_id, + recipient = %recipient_identity_id, + %error, + "private contact request paid for; reading it back failed, a later sync reads it" + ); + } + let recorded = self + .with_managed(sender_identity_id, |managed| { + managed + .dashpay() + .outgoing_request(recipient_identity_id) + .filter(|tracked| { + tracked.account_reference == expected.account_reference + && tracked.encrypted_public_key == expected.encrypted_public_key + }) + .cloned() + }) + .await?; + let Some(contact_request) = recorded else { + return Err(PlatformWalletError::PrivateContactRequestPending( + *recipient_identity_id, + )); + }; + if let Err(e) = self + .register_contact_account( + sender_identity_id, + recipient_identity_id, + account_index, + contact_xpub_ext, + ) + .await + { + tracing::warn!( + sender = %sender_identity_id, + recipient = %recipient_identity_id, + error = %e, + "private contact request sent; its receiving account is not registered yet, the \ + next sync registers it" + ); + } + Ok(contact_request) + } + + /// Accept a contact request that came privately, replying privately: as + /// [`Self::accept_contact_request_with_external_signer`], with the reply + /// sent by [`Self::send_private_contact_request`], signed by `signer`, + /// `pay` paying for it and `shielded_address` the address the contact may + /// pay us at privately. + /// + /// As for the send, the store is read to its end first when it was not + /// since load, so a reply paid for in an earlier process is adopted + /// rather than paid for again; while that read cannot finish, the accept + /// is refused with [`PlatformWalletError::PrivateContactRequestPending`]. + /// A reply whose payment is unconfirmed since before a restart holds the + /// pair from the moment the wallet binds to its shielded coordinator. + pub async fn accept_private_contact_request( + &self, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + pay: F, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + self.read_private_store_before_sending(&request.recipient_id, &request.sender_id, crypto) + .await?; + self.accept_contact_request_replying_with(request, crypto, || { + self.send_private_reply(request, shielded_address, signer, crypto, pay) + }) + .await + } + + /// Accept `request` replying the way it came: privately, paid by `pay`, + /// when it came privately and we have sent its sender no request yet + /// ([`Self::accept_private_contact_request`]); otherwise with a + /// `contactRequest` document, or by adopting the reply already sent + /// ([`Self::accept_contact_request_with_external_signer`]). The one accept + /// a host calls: which reply a request takes is the wallet's decision. + /// `pay` runs only for a private reply. + /// + /// The store is read to its end first if it was not since load, before + /// the reply is chosen, whatever the request: a reply paid for in an + /// earlier process, to a public request too, is then adopted, with no + /// payment and no public reply. While that read cannot finish, the accept + /// is refused with [`PlatformWalletError::PrivateContactRequestPending`]. + /// On a network that serves no private documents yet, nothing is read. + pub async fn accept_contact_request_replying_in_kind( + &self, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + pay: F, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + self.read_private_store_before_deciding(&request.recipient_id, &request.sender_id, crypto) + .await?; + if self.needs_private_reply(request).await { + self.accept_private_contact_request(request, shielded_address, signer, crypto, pay) + .await + } else { + self.accept_contact_request_with_external_signer(request, signer, crypto) + .await + } + } + + /// The wallet's identities a pass reads the store for (those with an HD + /// index, whose keys the signer can derive) and the paths of their keys: + /// every `ECDSA_SECP256K1` decryption or encryption key, disabled ones + /// included, as an entry sealed before a key was disabled still opens with + /// it, and every such encryption key as a sender. + async fn private_scan_identities( + &self, + ) -> Result<(Vec, PrivateDocumentKeyPaths), PlatformWalletError> { + let wm = self.wallet_manager.read().await; + let info = wm + .get_wallet_info(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + let mut identities = Vec::new(); + let mut paths = PrivateDocumentKeyPaths::default(); + for identity in info.identity_manager.all_identities() { + let identity_id = identity.id(); + let Some(managed) = info.identity_manager.managed_identity(&identity_id) else { + continue; + }; + let Some(identity_index) = managed.identity_index else { + continue; + }; + let path = |key_id| { + IdentityWallet::::identity_auth_derivation_path( + self.sdk.network, + KeyDerivationType::ECDSA, + identity_index, + key_id, + ) + }; + let mut recipient_paths = Vec::new(); + for (key_id, key) in managed.identity.public_keys() { + if key.key_type() != KeyType::ECDSA_SECP256K1 + || !recipient_key_purpose_is_valid(key.purpose()) + { + continue; + } + recipient_paths.push((*key_id, path(*key_id)?)); + if key.purpose() == Purpose::ENCRYPTION { + paths.senders.push((identity_id, *key_id, path(*key_id)?)); + } + } + if recipient_paths.is_empty() { + continue; + } + paths.recipients.push((identity_id, recipient_paths)); + identities.push(ScannedIdentity { + identity_id, + identity_index, + position: managed.dashpay().private_contact_requests_position(), + failed: false, + rewound: false, + key_fingerprint: managed.private_scan_key_fingerprint(), + }); + } + Ok((identities, paths)) + } + + /// Read DashPay's private contact request store for the wallet's + /// identities, at most `max_pages` pages from the earliest of their + /// cursors, and ingest what it holds for them: requests to them, each + /// checked against the keys of its sender and of the identity it is to, + /// and requests they sent, recovered from the store alone and checked to + /// be theirs. Contacts establish, and their DashPay accounts are queued, + /// as the document sync does. A request older than the one tracked with + /// the same contact, either way and from either channel, is passed over. + /// + /// A request that came privately carries no auto-accept proof into the + /// wallet: accepting it replies privately, which pays from the shielded + /// pool, so it is accepted by hand + /// ([`Self::accept_private_contact_request`]). + /// + /// Refused with [`PlatformWalletError::SeedMismatch`], before any page + /// is read and with no position moved, when `crypto` does not resolve + /// this wallet's seed ([`IdentityWallet::verify_seed_binds`]): another + /// seed derives valid but unrelated keys, which open nothing, so its pass + /// would record as read pages holding requests it never saw, and nothing + /// rewinds them once the provider is fixed. + pub async fn sync_private_contact_requests( + &self, + crypto: &C, + max_pages: usize, + ) -> Result + where + C: ContactCryptoProvider + Sync, + { + self.release_discharged_private_requests().await?; + self.reconcile_private_scan_keys().await?; + let (mut identities, paths) = self.private_scan_identities().await?; + let mut report = PrivateContactSyncReport::default(); + let Some(start) = identities.iter().map(|identity| identity.position).min() else { + report.complete = true; + return Ok(report); + }; + // Before anything is read or recorded: once a pass, as the keys it + // scans with all come from the same provider. + self.verify_seed_binds(crypto).await?; + let identity_indexes: BTreeMap = identities + .iter() + .map(|identity| (identity.identity_id, identity.identity_index)) + .collect(); + let key_path = |identity_id: &Identifier, key_id: KeyID| { + let identity_index = identity_indexes + .get(identity_id) + .copied() + .ok_or_else(|| PlatformWalletError::IdentityNotFound(*identity_id))?; + IdentityWallet::::identity_auth_derivation_path( + self.sdk.network, + KeyDerivationType::ECDSA, + identity_index, + key_id, + ) + }; + let contract = dashpay_contract()?; + // Fetched once a pass, and again when a later page's entries are newer + // than the proof: the senders and the wallet's own identities, whose + // keys a request is checked against as they were when it was appended. + let mut proved_identities: BTreeMap = BTreeMap::new(); + // The recipients of the requests the wallet's identities sent, read for + // the key id of the key a request is sealed to, as they were when it + // was appended or later. + let mut recipients: BTreeMap = BTreeMap::new(); + report.next_position = start; + let persists_scan_state = self.persister.persists_private_contact_scan_state(); + + // The pages are fetched many at once while earlier ones are processed, + // and scanned in batches of those ready, one key derivation a batch. + let mut batches = std::pin::pin!(self + .sdk + .private_document_pages( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + start, + MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES, + ) + .take(max_pages) + .ready_chunks(PRIVATE_SCAN_BATCH_PAGES)); + while let Some(batch) = batches.next().await { + // The pages before a failed fetch are processed, then the error + // ends the pass. + let mut fetched = Vec::with_capacity(batch.len()); + let mut failure = None; + for page in batch { + match page { + Ok(page) => fetched.push(page), + Err(error) => { + failure = Some(error); + break; + } + } + } + let scans = if fetched.is_empty() { + Vec::new() + } else { + let pages: Vec<(u64, &PrivateDocumentEntries)> = fetched + .iter() + .map(|page| (page.start_position, &page.entries)) + .collect(); + crypto + .scan_private_document_pages( + &paths, + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + &pages, + ) + .await? + }; + if scans.len() != fetched.len() { + return Err(PlatformWalletError::InvalidIdentityData(format!( + "the crypto provider sorted {} pages into {} scans", + fetched.len(), + scans.len() + ))); + } + let mut reached_end = false; + for scan in scans { + report.pages_read += 1; + let stale: BTreeMap = private_contact_request_identities(&scan) + .into_iter() + .filter(|(identity_id, height)| { + proved_identities + .get(identity_id) + .is_none_or(|proved| proved.proved_at_height < *height) + }) + .collect(); + proved_identities.extend( + self.sdk + .fetch_private_contact_request_identities(&stale) + .await?, + ); + let read = private_contact_requests_from_scan( + &contract, + scan, + &proved_identities, + self.sdk.version(), + )?; + // Each recipient is fetched on its own, so one that cannot be read + // holds back only the identities whose sent requests it is for. + let mut stale_recipients: BTreeMap = BTreeMap::new(); + for sent in &read.sent { + let height = stale_recipients.entry(sent.recipient_id).or_default(); + *height = (*height).max(sent.block_height); + } + stale_recipients.retain(|recipient_id, height| { + recipients + .get(recipient_id) + .is_none_or(|proved| proved.proved_at_height < *height) + }); + let mut unreadable_recipients = BTreeSet::new(); + for (recipient_id, height) in stale_recipients { + match self + .sdk + .fetch_private_contact_request_identities(&BTreeMap::from([( + recipient_id, + height, + )])) + .await + { + Ok(fetched) => recipients.extend(fetched), + Err(error) => { + tracing::warn!( + recipient = %recipient_id, + %error, + "the recipient of a sent private contact request could not be read; \ + its sender's cursor is held" + ); + unreadable_recipients.insert(recipient_id); + } + } + } + let held: BTreeSet = read + .sent + .iter() + .filter(|sent| unreadable_recipients.contains(&sent.recipient_id)) + .map(|sent| sent.sender_id) + .collect(); + + // The wallet's records of the page's requests, under the shared + // keys the signer computes. + let mut records: Vec<(Identifier, ContactRequest, bool, Option<[u8; 43]>)> = + Vec::new(); + // The entry commitment of each sent request's record: a pending + // request is settled by it once that record is reconciled. + let mut sent_commitments: Vec<([u8; 32], ContactRequest)> = Vec::new(); + for received in &read.received { + let Some(sender) = proved_identities + .get(&received.sender_id) + .and_then(|proved| proved.identity.as_ref()) + else { + continue; + }; + let sender_key = identity_public_key(sender, received.sender_key_id)?; + let shared_secret = crypto + .ecdh_shared_secret( + &key_path(&received.recipient_id, received.recipient_key_id)?, + &sender_key, + ) + .await?; + // Accepting replies privately, by hand: no auto-accept. + let mut record = private_request_record( + received.sender_id, + received.recipient_id, + received.sender_key_id, + received.recipient_key_id, + &received.request, + &received.commitment, + &shared_secret, + received.block_time_ms, + ); + record.auto_accept_proof = None; + records.push(( + received.recipient_id, + record, + true, + received.request.shielded_address, + )); + } + for sent in &read.sent { + if unreadable_recipients.contains(&sent.recipient_id) { + continue; + } + // The recipient's key the entry was sealed to, compared as + // a curve point: a key may hold the point compressed or + // uncompressed, and the sealer reads both. Only the keys an + // entry can be sealed to are matched. + let recipient_key_id = recipients + .get(&sent.recipient_id) + .and_then(|proved| proved.identity.as_ref()) + .and_then(|recipient| { + recipient + .public_keys() + .iter() + .find(|(_, key)| { + key.key_type() == KeyType::ECDSA_SECP256K1 + && recipient_key_purpose_is_valid(key.purpose()) + && PublicKey::from_slice(key.data().as_slice()) + .is_ok_and(|point| point == sent.recipient_public_key) + }) + .map(|(key_id, _)| *key_id) + }); + let Some(recipient_key_id) = recipient_key_id else { + tracing::warn!( + sender = %sent.sender_id, + recipient = %sent.recipient_id, + position = sent.position, + "sent private contact request is sealed to a key its recipient does not \ + hold; not recorded" + ); + continue; + }; + let shared_secret = crypto + .ecdh_shared_secret( + &key_path(&sent.sender_id, sent.sender_key_id)?, + &sent.recipient_public_key, + ) + .await?; + let record = private_request_record( + sent.sender_id, + sent.recipient_id, + sent.sender_key_id, + recipient_key_id, + &sent.request, + &sent.commitment, + &shared_secret, + sent.block_time_ms, + ); + sent_commitments.push((sent.commitment, record.clone())); + records.push((sent.sender_id, record, false, None)); + } + for rejected in &read.rejected { + tracing::debug!( + position = rejected.position, + reason = ?rejected.reason, + "private contact request refused" + ); + } + report.rejected.extend(read.rejected.iter().cloned()); + + // Ingest what the page held for each identity, then advance its + // cursor past the page if that reached disk. + let mut candidates = Vec::new(); + { + let mut wm = self.wallet_manager.write().await; + let (wallet, info) = wm + .get_wallet_mut_and_info_mut(&self.wallet_id) + .ok_or_else(|| { + PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)) + })?; + for identity in identities.iter_mut() { + let identity_id = identity.identity_id; + let Some(managed) = + info.identity_manager.managed_identity_mut(&identity_id) + else { + continue; + }; + if held.contains(&identity_id) && !identity.failed { + identity.failed = true; + report.unpersisted_identities.push(identity_id); + } + let mut received = Vec::new(); + let mut addresses = Vec::new(); + let mut sent = Vec::new(); + for (owner, record, is_received, shielded_address) in &records { + if *owner != identity_id { + continue; + } + if *is_received { + addresses.push((record.clone(), *shielded_address)); + received.push(record.clone()); + } else { + // Our receiving account for the contact is + // scanned from the wallet's birth. + managed.note_sent_request_core_height(record.recipient_id, 0); + sent.push(record.clone()); + } + } + let mut rotated = Vec::new(); + let mut ingested = Vec::new(); + let received_ok = ingest_received_requests( + managed, + &self.persister, + identity_id, + newest_received_per_sender(received), + &mut rotated, + &mut ingested, + ); + let sent_ok = ingest_sent_requests( + managed, + &self.persister, + identity_id, + newest_sent_per_recipient(sent), + &mut ingested, + ); + report.requests.extend(ingested); + // The address of the request tracked from each contact, + // once ingest decided which that is. + for (record, shielded_address) in &addresses { + managed.note_private_shielded_address(record, *shielded_address); + } + // The addresses are stored before the read position + // that covers the entries they came from. + let mut stored = received_ok && sent_ok; + if stored && persists_scan_state && !addresses.is_empty() { + let addresses_changeset = + shielded_address_changeset(managed, identity_id, &addresses); + if let Err(error) = self.persister.store(addresses_changeset.into()) { + tracing::error!( + owner = %identity_id, %error, + "private contacts' shielded addresses not stored; leaving \ + the private cursor for retry" + ); + stored = false; + } + } + if stored { + let unresolved = managed.dashpay().private_payments_unresolved(); + // A pending private request is settled once its + // entry's record is reconciled: the request tracked + // to the recipient is that record, carries its + // account reference (another copy of it), or is + // newer. An entry whose record was not built, its + // recipient unreadable, settles nothing. + let mut read_back = Vec::new(); + for (commitment, record) in sent_commitments + .iter() + .filter(|(_, record)| record.sender_id == identity_id) + { + let reconciled = managed + .dashpay() + .outgoing_request(&record.recipient_id) + .is_some_and(|tracked| { + tracked.account_reference == record.account_reference + || predates(record, tracked) + }); + if reconciled { + managed.settle_private_reply(&record.recipient_id, commitment); + read_back.push(*commitment); + } + } + let before = managed.dashpay().private_contact_requests_position(); + identity.advance(managed, read.next_position); + let after = managed.dashpay().private_contact_requests_position(); + // Stored once it moved, or once the last private + // payment it held settled. + if persists_scan_state + && (after != before + || managed.dashpay().private_payments_unresolved() + != unresolved) + { + store_private_position( + &self.persister, + identity_id, + managed.private_contact_requests_position_record(), + ); + } + // The shielded store's journal records of the entries + // read back are discharged once what records them is + // durable: a record lost with them would leave the + // pair unheld while no request of it is recorded. + if !read_back.is_empty() { + note_read_back_once_durable(&self.persister, read_back); + } + } else if !identity.failed { + identity.failed = true; + report.unpersisted_identities.push(identity_id); + } + // Read to the store's end with nothing held back this + // pass: every entry was ingested for this identity. + // Not once its cursor was rewound under the pass (a + // sender unignored): the entries before it were read + // with the sender ignored, and the next pass reads + // them again. + if read.is_complete() + && !identity.failed + && !identity.rewound + && identity.keys_current(managed) + { + managed.note_private_contact_requests_read(); + } + tear_down_stale_external_accounts(wallet, info, &identity_id, rotated); + candidates.push(( + identity_id, + Self::collect_account_build_candidates(info, &identity_id), + )); + } + } + for (identity_id, identity_candidates) in candidates { + for candidate in identity_candidates { + self.build_contact_accounts(&identity_id, candidate).await; + } + } + + report.next_position = read.next_position; + if read.is_complete() { + // A pass whose ingest did not reach disk for an identity is + // not complete: that identity reads its entries again. + report.complete = report.unpersisted_identities.is_empty(); + reached_end = true; + break; + } + } + if let Some(error) = failure { + return Err(error.into()); + } + if reached_end { + break; + } + } + Ok(report) + } + + /// Restore, after a restart, the pending private requests whose payment + /// is unconfirmed: `entries` are those of the signed transitions the + /// shielded store keeps for unconfirmed spends to DashPay's + /// `privateContactRequest` store + /// (`PlatformWallet::restore_pending_private_contact_requests` reads + /// them). Each opens with the keys of the wallet's identities, and every + /// one an identity sent reserves its recipient again under the entry's + /// commitment, so no second request to it is sent, privately or + /// publicly, until a private sync reads the entry back and settles it. + /// A recipient already reserved keeps its reservation, which gets the + /// entry's commitment only when it has none yet. Returns the (sender, + /// recipient) pairs restored. + /// + /// An entry is discharged once its own request or a superseding one is + /// recorded: it is left alone when the request the sender tracks to its + /// recipient has the entry's account reference, or a later version. So an + /// unconfirmed rotation is restored over the request it rotates, and an + /// entry read back while its spend still waits for confirmation is not. + /// + /// An entry whose request does not decode is left alone: no sync could + /// record it, so its reservation would never settle. A record saved with + /// the request it pays for is restored without this call when the wallet + /// binds to its shielded coordinator + /// (`PlatformWallet::restore_private_contact_payments`); this call stays + /// for records saved before that. + pub async fn restore_pending_private_contact_requests( + &self, + entries: &[BinaryData], + crypto: &C, + ) -> Result, PlatformWalletError> + where + C: ContactCryptoProvider + Sync, + { + self.restore_private_reservations(entries, crypto, None) + .await + } + + /// Restore, after a restart or a bind, the reservations of the private + /// contact request payments the shielded store's private payment journal + /// keeps for this wallet, `payments` being each record's (sender, + /// recipient, account reference, the entry's commitment), and release the + /// holds a load set on the wallet's identities, all under one wallet + /// lock: an identity added in between is either restored and released or + /// neither. Nothing is opened, so no keys are needed. + /// + /// A payment whose request the sender's tracked request to that recipient + /// records or supersedes is resolved and left alone, as + /// [`Self::restore_pending_private_contact_requests`] leaves it. One + /// whose sender the wallet does not manage holds its pair for the public + /// send of an identity added later, until the next restoration + /// (`WalletPersister::hold_absent_sender_private_pairs`). + /// + /// An identity loaded marking a private payment that may be unresolved + /// is released when the journal is authoritative (`journal_authoritative`: + /// open before this bind, so it keeps every payment the wallet made since) + /// or holds a restored reservation now. Otherwise the journal cannot + /// account for its payment, the store being new or replaced: it stays + /// held, and the restoration fails with + /// [`PlatformWalletError::ShieldedRecoveryCorrupted`]. Returns the + /// (sender, recipient) pairs restored. + #[cfg(all(test, feature = "shielded"))] + pub(crate) async fn restore_private_payments( + &self, + payments: &[JournaledPrivatePayment], + journal_authoritative: bool, + ) -> Result, PlatformWalletError> { + restore_journaled_private_payments( + &self.wallet_manager, + &self.generation, + &self.persister, + self.wallet_id, + payments, + journal_authoritative, + ) + .await + } + + /// Whether an earlier private payment from `sender` to `recipient` that + /// the shielded store's journal keeps, `journaled` being each one's + /// (account reference, entry commitment), may still execute, neither its + /// request nor a later one being recorded: then the recipient stays + /// reserved for that entry instead of the one about to be paid, which + /// must not be. + #[cfg(feature = "shielded")] + pub(crate) async fn hold_for_journaled_private_payment( + &self, + sender: &Identifier, + recipient: &Identifier, + journaled: &[(u32, [u8; 32])], + ) -> Result { + if journaled.is_empty() { + return Ok(false); + } + let payments: Vec = journaled + .iter() + .map(|&(account_reference, commitment)| { + (*sender, *recipient, account_reference, commitment) + }) + .collect(); + let mut wm = self.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + if info.identity_manager.managed_identity(sender).is_none() { + return Err(PlatformWalletError::IdentityNotFound(*sender)); + } + let reserved = reserve_unresolved_private_payments( + info, + &self.generation, + self.wallet_id, + &self.persister, + &payments, + Some((*sender, *recipient)), + )?; + Ok(!reserved.restored.is_empty()) + } + + /// Whether a request from `sender` to `recipient` among `entries`, paid + /// for before a restart and unconfirmed, may still execute, neither it nor + /// a later request being recorded (a rotation over the tracked request + /// counts): then the recipient stays reserved for that entry instead of + /// the one about to be paid, which must not be. The other entries are + /// restored as [`Self::restore_pending_private_contact_requests`] restores + /// them. + #[cfg(test)] + pub(crate) async fn hold_for_restored_private_contact_request( + &self, + entries: &[BinaryData], + crypto: &C, + sender: &Identifier, + recipient: &Identifier, + ) -> Result + where + C: ContactCryptoProvider + Sync, + { + let pair = (*sender, *recipient); + Ok(self + .restore_private_reservations(entries, crypto, Some(pair)) + .await? + .contains(&pair)) + } + + /// Open `entries` as one page of the store and reserve the recipient of + /// each the wallet's identities sent whose request is neither recorded + /// nor superseded yet; the pair `paying` is retargeted to its restored + /// entry instead. Reserved through [`reserve_unresolved_private_payments`], + /// as a journal restoration reserves: fenced to this handle's wallet + /// generation, and leaving alone an entry whose payment's journal record + /// was discharged in this session, though the discharge came while the + /// entries were opened (checked under the wallet lock). + async fn restore_private_reservations( + &self, + entries: &[BinaryData], + crypto: &C, + paying: Option<(Identifier, Identifier)>, + ) -> Result, PlatformWalletError> + where + C: ContactCryptoProvider + Sync, + { + if entries.is_empty() { + return Ok(Vec::new()); + } + let (_, paths) = self.private_scan_identities().await?; + if paths.senders.is_empty() { + return Ok(Vec::new()); + } + let contract = dashpay_contract()?; + let document_type = contract + .document_type_for_name(PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME) + .map_err(|error| { + PlatformWalletError::InvalidIdentityData(format!( + "the bundled DashPay has no {PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME} type: \ + {error}" + )) + })?; + let page = PrivateDocumentEntries { + chunk_power: None, + entries: entries + .iter() + .zip(0u64..) + .map(|(data, position)| PrivateDocumentEntry { + position, + block_height: 0, + block_time_ms: 0, + data: data.to_vec(), + }) + .collect(), + total_count: entries.len() as u64, + }; + let scan = crypto + .scan_private_document_page( + &paths, + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + &page, + ) + .await?; + + let mut payments: Vec = Vec::new(); + for sent in scan.sent { + let sender = sent.sender_identity_id; + // The wallet's own entries name the identity that sealed them. + if Identifier::from(*sent.entry.sender_identity_id()) != sender { + continue; + } + let recipient = Identifier::from(*sent.entry.recipient_identity_id()); + let request = decode_private_document_payload( + document_type, + sent.entry.payload(), + self.sdk.version(), + ) + .map_err(|error| error.to_string()) + .and_then(|properties| { + PrivateContactRequest::from_properties(properties) + .map_err(|error| error.to_string()) + }); + let request = match request { + Ok(request) => request, + Err(error) => { + tracing::warn!( + sender = %sender, + recipient = %recipient, + %error, + "an unconfirmed private contact request does not decode; not restored" + ); + continue; + } + }; + payments.push(( + sender, + recipient, + request.account_reference, + *sent.entry.commitment(), + )); + } + let mut wm = self.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&self.wallet_id) + .ok_or_else(|| PlatformWalletError::WalletNotFound(hex::encode(self.wallet_id)))?; + // A sender the wallet does not manage is not held here: the journal + // restoration holds those. + let reserved = reserve_unresolved_private_payments( + info, + &self.generation, + self.wallet_id, + &self.persister, + &payments, + paying, + )?; + Ok(reserved.restored) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::changeset::{ + ClientStartState, ClientWalletStartState, IdentityManagerStartState, + PersistenceCapabilities, PersistenceError, PlatformWalletChangeSet, + PlatformWalletPersistence, + }; + use crate::events::{EventHandler, PlatformEventHandler}; + use crate::wallet::identity::network::identity_auth_derivation_path_for_type; + use crate::wallet::identity::network::SeedCryptoProvider; + use crate::wallet::platform_wallet::{PlatformWallet, WalletId}; + #[cfg(feature = "shielded")] + use crate::wallet::shielded::coordinator::{ + private_payment_journal_snapshot, PrunedAnchorEvidence, + }; + #[cfg(feature = "shielded")] + use crate::wallet::shielded::operations::arm_redrive_before_broadcast; + #[cfg(feature = "shielded")] + use crate::wallet::shielded::{ + FileBackedShieldedStore, NetworkShieldedCoordinator, OrchardKeySet, PrivateContactPayment, + ShieldedNote, ShieldedStore, SubwalletId, + }; + use crate::PlatformWalletManager; + use dash_sdk::error::ContextProviderError; + use dash_sdk::platform::private_document_entries::{ + PrivateDocumentEntriesQuery, PrivateDocumentEntry, + }; + use dash_sdk::platform::private_document_scan::PrivateDocumentScan; + use dash_sdk::platform::ContextProvider; + use dpp::address_funds::AddressWitness; + use dpp::data_contract::{DataContract, TokenConfiguration}; + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeySettersV0; + use dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; + use dpp::identity::v0::IdentityV0; + use dpp::identity::{IdentityPublicKey, SecurityLevel}; + use dpp::version::PlatformVersion; + use dpp::ProtocolError; + use key_wallet::account::account_collection::DashpayAccountKey; + use key_wallet::mnemonic::Mnemonic; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + use key_wallet::Network; + use platform_encryption::{ + account_reference_version, decrypt_account_label, decrypt_extended_public_key, + }; + #[cfg(feature = "shielded")] + use std::collections::HashSet; + use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; + use std::sync::Mutex; + use std::time::Duration; + #[cfg(feature = "shielded")] + use tokio_util::sync::CancellationToken; + + const ALICE_PHRASE: &str = "abandon abandon abandon abandon abandon abandon abandon abandon \ + abandon abandon abandon about"; + const BOB_PHRASE: &str = + "legal winner thank year wave sausage worth useful legal winner thank yellow"; + const ALICE: [u8; 32] = [0xA1; 32]; + const BOB: [u8; 32] = [0xB1; 32]; + const CAROL_PHRASE: &str = + "letter advice cage absurd amount doctor acoustic avoid letter advice cage above"; + const CAROL: [u8; 32] = [0xC2; 32]; + /// The time of the block that appends the test entries. + const BLOCK_TIME_MS: u64 = 1_700_000_000_000; + /// The key ids of each test identity's signing key and DashPay keys. + const SIGNING_KEY_ID: KeyID = 0; + const ENCRYPTION_KEY_ID: KeyID = 1; + const DECRYPTION_KEY_ID: KeyID = 2; + + #[derive(Default)] + /// A backend that keeps nothing, declaring that it keeps the private + /// contact request scan state and commits each store inline, so private + /// payments, which need their recovery marker kept, run over it. The tests + /// using it never reload what it kept. + struct NoPersistence; + + impl PlatformWalletPersistence for NoPersistence { + fn persistence_capabilities(&self) -> PersistenceCapabilities { + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART + } + + fn store_commits_inline(&self) -> bool { + true + } + + fn store( + &self, + _wallet_id: WalletId, + _changeset: PlatformWalletChangeSet, + ) -> Result<(), PersistenceError> { + Ok(()) + } + + fn flush(&self, _wallet_id: WalletId) -> Result<(), PersistenceError> { + Ok(()) + } + + fn load(&self) -> Result { + Ok(ClientStartState::default()) + } + } + + struct NoopEventHandler; + impl EventHandler for NoopEventHandler {} + impl PlatformEventHandler for NoopEventHandler {} + + /// A context provider holding the bundled DashPay contract. + struct DashpayContractProvider(Arc); + + impl ContextProvider for DashpayContractProvider { + fn get_data_contract( + &self, + id: &Identifier, + _platform_version: &PlatformVersion, + ) -> Result>, ContextProviderError> { + Ok((*id == self.0.id()).then(|| Arc::clone(&self.0))) + } + + fn get_token_configuration( + &self, + _token_id: &Identifier, + ) -> Result, ContextProviderError> { + Ok(None) + } + + fn get_quorum_public_key( + &self, + _quorum_type: u32, + _quorum_hash: [u8; 32], + _core_chain_locked_height: u32, + ) -> Result<[u8; 48], ContextProviderError> { + Err(ContextProviderError::Config( + "no quorum keys in this test".into(), + )) + } + + fn get_platform_activation_height(&self) -> Result { + Ok(1) + } + } + + fn seed(phrase: &str) -> [u8; 64] { + *Mnemonic::from_phrase(phrase) + .expect("test mnemonic") + .to_seed("") + } + + /// The secret key `key_id` of identity index 0 of `phrase`'s seed. + fn identity_secret_key(phrase: &str, key_id: KeyID) -> SecretKey { + let wallet = key_wallet::wallet::Wallet::from_seed_bytes( + seed(phrase), + Network::Testnet, + WalletAccountCreationOptions::None, + ) + .expect("test wallet"); + let path = identity_auth_derivation_path_for_type( + Network::Testnet, + KeyDerivationType::ECDSA, + 0, + key_id, + ) + .expect("identity key path"); + let xprv = wallet + .derive_extended_private_key(&path) + .expect("identity key"); + SecretKey::from_secret_bytes(*xprv.private_key.to_bytes()).expect("secret key") + } + + /// The identity `id` whose keys are those of identity index 0 of + /// `phrase`'s seed: its HIGH signing key 0, encryption key 1 and + /// decryption key 2. + fn identity(phrase: &str, id: [u8; 32]) -> Identity { + let key = |key_id: KeyID, purpose: Purpose, security_level: SecurityLevel| { + IdentityPublicKey::V0(IdentityPublicKeyV0 { + id: key_id, + purpose, + security_level, + contract_bounds: None, + key_type: KeyType::ECDSA_SECP256K1, + read_only: false, + data: BinaryData::new( + PublicKey::from_secret_key(&identity_secret_key(phrase, key_id)) + .serialize() + .to_vec(), + ), + disabled_at: None, + }) + }; + let keys = [ + key(SIGNING_KEY_ID, Purpose::AUTHENTICATION, SecurityLevel::HIGH), + key( + ENCRYPTION_KEY_ID, + Purpose::ENCRYPTION, + SecurityLevel::MEDIUM, + ), + key( + DECRYPTION_KEY_ID, + Purpose::DECRYPTION, + SecurityLevel::MEDIUM, + ), + ]; + Identity::V0(IdentityV0 { + id: Identifier::from(id), + public_keys: keys.into_iter().map(|key| (key.id(), key)).collect(), + balance: 0, + revision: 0, + }) + } + + /// The signer holding `identity`'s signing key, as a host's signer holds + /// the keys it signs transitions with. + fn signer(phrase: &str, identity: &Identity) -> simple_signer::signer::SimpleSigner { + let mut signer = simple_signer::signer::SimpleSigner::default(); + signer.add_identity_public_key( + identity + .get_public_key_by_id(SIGNING_KEY_ID) + .expect("the signing key") + .clone(), + identity_secret_key(phrase, SIGNING_KEY_ID).to_secret_bytes(), + ); + signer + } + + /// A backend recording every changeset stored, attesting `capabilities`. + #[derive(Default)] + struct RecordingPersistence { + capabilities: PersistenceCapabilities, + stored: Mutex>, + } + + impl RecordingPersistence { + fn attesting(capabilities: PersistenceCapabilities) -> Arc { + Arc::new(Self { + capabilities, + stored: Mutex::new(Vec::new()), + }) + } + + /// The contact changesets stored, in order. + fn contacts(&self) -> Vec { + self.stored + .lock() + .expect("stored") + .iter() + .filter_map(|changeset| changeset.contacts.clone()) + .collect() + } + } + + impl PlatformWalletPersistence for RecordingPersistence { + fn store( + &self, + _wallet_id: WalletId, + changeset: PlatformWalletChangeSet, + ) -> Result<(), PersistenceError> { + self.stored.lock().expect("stored").push(changeset); + Ok(()) + } + + fn flush(&self, _wallet_id: WalletId) -> Result<(), PersistenceError> { + Ok(()) + } + + fn load(&self) -> Result { + Ok(ClientStartState::default()) + } + + fn persistence_capabilities(&self) -> PersistenceCapabilities { + self.capabilities + } + } + + /// A wallet of `phrase` holding `own` at identity index 0, on a mock SDK that + /// answers identity fetches with `known` and page 0 of DashPay's private + /// contact request store with `entries`. + async fn wallet( + phrase: &str, + own: &Identity, + known: &[&Identity], + entries: Vec>, + ) -> (PlatformWalletManager, Arc) { + wallet_on(Arc::new(NoPersistence), phrase, own, known, entries).await + } + + /// [`wallet`] on the backend `persistence`. + async fn wallet_on( + persistence: Arc

, + phrase: &str, + own: &Identity, + known: &[&Identity], + entries: Vec>, + ) -> (PlatformWalletManager

, Arc) { + wallet_at( + persistence, + phrase, + own, + known, + entries, + PlatformVersion::latest(), + None, + ) + .await + } + + /// [`wallet_on`], with its SDK pinned to `platform_version` and holding + /// DashPay as that version bundles it, which a DashPay fetch also + /// answers; `verified_network` is the network's version a verified + /// answer showed, when one did. + async fn wallet_at( + persistence: Arc

, + phrase: &str, + own: &Identity, + known: &[&Identity], + entries: Vec>, + platform_version: &'static PlatformVersion, + verified_network: Option, + ) -> (PlatformWalletManager

, Arc) { + let sdk = dashpay_sdk(known, entries, platform_version, verified_network).await; + let manager = + PlatformWalletManager::new(Arc::new(sdk), persistence, Arc::new(NoopEventHandler)); + let wallet = manager + .create_wallet_from_seed_bytes( + Network::Testnet, + &seed(phrase), + WalletAccountCreationOptions::Default, + Some(0), + ) + .await + .expect("wallet"); + { + let iw = wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let info = wm + .get_wallet_info_mut(&wallet.wallet_id()) + .expect("wallet info"); + info.identity_manager + .add_identity(own.clone(), 0, wallet.wallet_id(), &iw.persister) + .expect("own identity"); + } + (manager, wallet) + } + + /// The mock SDK of [`wallet_at`]: pinned to `platform_version`, holding + /// DashPay as that version bundles it, answering fetches of `known` and + /// of the private store's first page, holding `entries`. + async fn dashpay_sdk( + known: &[&Identity], + entries: Vec>, + platform_version: &'static PlatformVersion, + verified_network: Option, + ) -> dash_sdk::Sdk { + let contract = Arc::new( + dpp::system_data_contracts::load_system_data_contract( + dpp::data_contracts::SystemDataContract::Dashpay, + platform_version, + ) + .expect("bundled DashPay"), + ); + let mut sdk = dash_sdk::SdkBuilder::new_mock() + .with_network(Network::Testnet) + .with_version(platform_version) + .with_context_provider(DashpayContractProvider(Arc::clone(&contract))) + .build() + .expect("mock sdk"); + if let Some(verified) = verified_network { + sdk.mock_verified_protocol_version(verified) + .expect("a mock SDK"); + } + sdk.mock() + .expect_fetch(contract.id(), Some((*contract).clone())) + .await + .expect("DashPay expectation"); + for identity in known { + sdk.mock() + .expect_fetch(identity.id(), Some((*identity).clone())) + .await + .expect("identity expectation"); + } + let total_count = entries.len() as u64; + let page = PrivateDocumentEntries { + chunk_power: None, + entries: entries + .into_iter() + .zip(0u64..) + // The mock proves identities at height 0, which an entry's + // identities must be proved at or after. + .map(|(data, position)| PrivateDocumentEntry { + position, + block_height: 0, + block_time_ms: BLOCK_TIME_MS, + data, + }) + .collect(), + total_count, + }; + sdk.mock() + .expect_fetch( + PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + ), + Some(page), + ) + .await + .expect("page expectation"); + sdk + } + + /// A backend handing back, on every load, a wallet as [`snapshot`] or + /// [`scan_state_snapshot`] took it: the relaunch of a wallet from an + /// earlier process, with the viewing key its shielded account 0 bound + /// with, when it bound one. It keeps what the relaunched wallet stores. + struct SnapshotPersistence { + wallet: key_wallet::Wallet, + core: key_wallet::wallet::ManagedWalletInfo, + identities: BTreeMap, + capabilities: PersistenceCapabilities, + #[cfg(feature = "shielded")] + viewing_key: Option<[u8; 96]>, + stored: Mutex>, + } + + impl SnapshotPersistence { + /// Whether the last position record stored for `owner` marks a private + /// payment that may be unresolved, if one was stored. + #[cfg(feature = "shielded")] + fn last_marker(&self, owner: &Identifier) -> Option { + last_marker(&self.stored.lock().expect("stored"), owner) + } + } + + /// Whether the last of `stored` to carry `owner`'s position record marks a + /// private payment that may be unresolved. + fn last_marker(stored: &[PlatformWalletChangeSet], owner: &Identifier) -> Option { + stored + .iter() + .rev() + .filter_map(|changeset| changeset.contacts.as_ref()) + .find_map(|contacts| contacts.private_contact_requests_positions.get(owner)) + .map(|position| position.private_payments_unresolved) + } + + impl PlatformWalletPersistence for SnapshotPersistence { + fn persistence_capabilities(&self) -> PersistenceCapabilities { + self.capabilities + } + + fn store( + &self, + _wallet_id: WalletId, + changeset: PlatformWalletChangeSet, + ) -> Result<(), PersistenceError> { + self.stored.lock().expect("stored").push(changeset); + Ok(()) + } + + fn flush(&self, _wallet_id: WalletId) -> Result<(), PersistenceError> { + Ok(()) + } + + fn load(&self) -> Result { + let wallet_id = self.wallet.compute_wallet_id(); + let mut wallet_identities = BTreeMap::new(); + wallet_identities.insert(wallet_id, self.identities.clone()); + let mut wallets = BTreeMap::new(); + wallets.insert( + wallet_id, + ClientWalletStartState { + wallet: self.wallet.clone(), + wallet_info: self.core.clone(), + identity_manager: IdentityManagerStartState { + wallet_identities, + ..Default::default() + }, + unused_asset_locks: BTreeMap::new(), + unconfirmed_outgoing_txs: Vec::new(), + dashpay_backfill: Default::default(), + }, + ); + #[allow(unused_mut)] + let mut state = ClientStartState { + wallets, + ..Default::default() + }; + #[cfg(feature = "shielded")] + if let Some(viewing_key) = self.viewing_key { + state + .shielded + .viewing_keys + .insert(SubwalletId::new(wallet_id, 0), viewing_key.to_vec()); + } + Ok(state) + } + } + + /// `wallet`, from `phrase`, with its identity `owner`, as a backend keeps + /// it once its shielded account 0 is bound. + #[cfg(feature = "shielded")] + async fn snapshot( + wallet: &PlatformWallet, + owner: &Identifier, + phrase: &str, + ) -> SnapshotPersistence { + let iw = wallet.identity(); + let wm = iw.wallet_manager.read().await; + let wallet_id = wallet.wallet_id(); + let info = wm.get_wallet_info(&wallet_id).expect("wallet info"); + let mut identities = BTreeMap::new(); + identities.insert( + 0, + info.identity_manager + .managed_identity(owner) + .expect("the owner") + .clone(), + ); + SnapshotPersistence { + wallet: wm.get_wallet(&wallet_id).expect("the wallet").clone(), + core: info.core_wallet.clone(), + identities, + capabilities: PersistenceCapabilities::SHIELDED_FVK_RESTART, + viewing_key: Some( + OrchardKeySet::from_seed(&seed(phrase), Network::Testnet, 0) + .expect("shielded keys") + .viewing_keys() + .to_fvk_bytes(), + ), + stored: Mutex::default(), + } + } + + /// `wallet`, with its identity `owner`, as a backend keeping the private + /// contact request scan state keeps it, with no shielded viewing key: the + /// identity's position record marks a private payment that may be + /// unresolved when `unresolved`, as a process that paid for one and ended + /// before it resolved stored it. + async fn scan_state_snapshot( + wallet: &PlatformWallet, + owner: &Identifier, + unresolved: bool, + ) -> SnapshotPersistence { + let iw = wallet.identity(); + let wm = iw.wallet_manager.read().await; + let wallet_id = wallet.wallet_id(); + let info = wm.get_wallet_info(&wallet_id).expect("wallet info"); + let mut managed = info + .identity_manager + .managed_identity(owner) + .expect("the owner") + .clone(); + managed.apply_private_contact_requests_position(PrivateContactRequestsPosition { + private_payments_unresolved: unresolved, + ..managed.private_contact_requests_position_record() + }); + SnapshotPersistence { + wallet: wm.get_wallet(&wallet_id).expect("the wallet").clone(), + core: info.core_wallet.clone(), + identities: BTreeMap::from([(0, managed)]), + capabilities: PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + #[cfg(feature = "shielded")] + viewing_key: None, + stored: Mutex::default(), + } + } + + /// A process that paid for a private contact request and ended before the + /// payment resolved stored the identity's position record marking it, in a + /// build with the `shielded` feature: reloaded, in any build, the identity + /// sends and accepts no public contact request, with nothing read, signed + /// or sent, as no reservation holds the pair the payment is for. A build + /// with the `shielded` feature releases it once the shielded store is + /// configured, which restores the reservations and stores the record + /// unmarked, when the store's journal was open before, so it accounts for + /// the payment: one opened only now (a deleted or replaced store) keeps + /// the identity held. No private sync releases it, and a build without + /// the `shielded` feature keeps it. + #[tokio::test] + async fn should_hold_an_identity_reloaded_marking_an_unresolved_private_payment() { + for journal_open_before in [false, true] { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let incoming = public_request(&bob, &alice, 3); + let (_before_manager, before) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + add_incoming(&before, ALICE, &incoming).await; + let kept = Arc::new(scan_state_snapshot(&before, &alice.id(), true).await); + + let manager = PlatformWalletManager::new( + Arc::new( + dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await, + ), + Arc::clone(&kept), + Arc::new(NoopEventHandler), + ); + manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + let restarted = manager + .get_wallet(&before.wallet_id()) + .await + .expect("the relaunched wallet"); + assert!(restarted.private_payments_held().await); + + let public = restarted + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{public:?}" + ); + let accepted = restarted + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!( + accepted, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{accepted:?}" + ); + + // No private request is paid for either. + let private = restarted + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!( + private, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{private:?}" + ); + // A private sync reads the store, and restores nothing. + let view = restarted.identity().dashpay(); + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the private sync"); + assert_eq!( + view.private_contact_readiness(&alice.id()) + .await + .expect("the readiness"), + PrivateContactReadiness { + private_payments_restored: false, + private_store_read: true, + unresolved_private_payment: true, + } + ); + assert!(restarted.private_payments_held().await, "still held"); + + #[cfg(feature = "shielded")] + { + let dir = tempfile::tempdir().expect("temporary shielded store"); + let path = dir.path().join("tree.sqlite"); + if journal_open_before { + // An earlier bind's restoration opened the wallet's journal. + let mut store = + FileBackedShieldedStore::open_path(&path, 100).expect("the store"); + store + .open_private_payment_journal(before.wallet_id()) + .expect("opened"); + } + manager + .configure_shielded(path) + .await + .expect("the shielded store is configured"); + if journal_open_before { + assert!(!restarted.private_payments_held().await, "restored"); + assert_eq!(kept.last_marker(&alice.id()), Some(false)); + assert_eq!( + view.private_contact_readiness(&alice.id()) + .await + .expect("the readiness"), + PrivateContactReadiness { + private_payments_restored: true, + private_store_read: true, + unresolved_private_payment: false, + } + ); + } else { + // A journal opened only now, as in a deleted or replaced + // store, cannot show the marked payment resolved. + assert!(restarted.private_payments_held().await, "still held"); + assert_ne!(kept.last_marker(&alice.id()), Some(false), "never cleared"); + let public = restarted + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{public:?}" + ); + } + } + #[cfg(not(feature = "shielded"))] + assert!( + restarted.private_payments_held().await, + "no shielded store restores it here, journal open before: {journal_open_before}" + ); + } + } + + /// A private payment is marked unresolved in its identity's position + /// record before it is paid, for a backend keeping the scan state, and + /// unmarked once its entry is read back; one released before anything was + /// paid is unmarked too. + #[tokio::test] + async fn should_mark_a_private_payment_unresolved_until_it_resolves() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let attesting = || { + RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + ) + }; + let marker = |persistence: &RecordingPersistence| { + last_marker(&persistence.stored.lock().expect("stored"), &alice.id()) + }; + + let persistence = attesting(); + let (_manager, alice_wallet) = wallet_on( + Arc::clone(&persistence), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let marked_when_paid = Arc::new(AtomicBool::new(false)); + let kept = Arc::new(Mutex::new(Vec::new())); + let pay = { + let paying = append_entry(&alice_wallet, 0, vec![], &kept); + let (persistence, marked, alice_id) = ( + Arc::clone(&persistence), + Arc::clone(&marked_when_paid), + alice.id(), + ); + move |entry: BinaryData, started: PaymentStarted| { + let unresolved = + last_marker(&persistence.stored.lock().expect("stored"), &alice_id); + marked.store(unresolved == Some(true), Ordering::SeqCst); + paying(entry, started) + } + }; + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ) + .await + .expect("paid and read back"); + assert!( + marked_when_paid.load(Ordering::SeqCst), + "marked before paying" + ); + assert_eq!(marker(&persistence), Some(false), "unmarked once read back"); + + let persistence = attesting(); + let (_manager, alice_wallet) = wallet_on( + Arc::clone(&persistence), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let marked_when_paid = Arc::new(AtomicBool::new(false)); + let failing = { + let (persistence, marked, alice_id) = ( + Arc::clone(&persistence), + Arc::clone(&marked_when_paid), + alice.id(), + ); + move |_: BinaryData, _: PaymentStarted| { + let unresolved = + last_marker(&persistence.stored.lock().expect("stored"), &alice_id); + marked.store(unresolved == Some(true), Ordering::SeqCst); + std::future::ready(Err::( + PlatformWalletError::InvalidParameter("no notes to pay with".to_string()), + )) + } + }; + let refused = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + failing, + ) + .await; + assert!(refused.is_err(), "{refused:?}"); + assert!( + marked_when_paid.load(Ordering::SeqCst), + "marked before paying" + ); + assert_eq!(marker(&persistence), Some(false), "unmarked once released"); + } + + /// A backend that buffers what is stored until a flush makes it durable, + /// declaring the private contact request scan state. Its flushes fail + /// once `flush_fails` is set. + #[derive(Default)] + struct BufferedPersistence { + buffered: Mutex>, + durable: Mutex>, + flush_fails: AtomicBool, + } + + impl PlatformWalletPersistence for BufferedPersistence { + fn persistence_capabilities(&self) -> PersistenceCapabilities { + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART + } + + fn store( + &self, + _wallet_id: WalletId, + changeset: PlatformWalletChangeSet, + ) -> Result<(), PersistenceError> { + self.buffered.lock().expect("buffered").push(changeset); + Ok(()) + } + + fn flush(&self, _wallet_id: WalletId) -> Result<(), PersistenceError> { + if self.flush_fails.load(Ordering::SeqCst) { + return Err(PersistenceError::backend("the disk is full")); + } + let mut buffered = self.buffered.lock().expect("buffered"); + self.durable + .lock() + .expect("durable") + .extend(buffered.drain(..)); + Ok(()) + } + + fn load(&self) -> Result { + Ok(ClientStartState::default()) + } + } + + /// A backend that hands back an identity's position record marking an + /// unresolved private payment without attesting the scan state any more, + /// as one whose host stopped declaring it, has its scan state read again + /// from the start, and keeps the hold: the payment may still execute. + #[tokio::test] + async fn should_keep_a_restored_hold_for_a_backend_that_no_longer_attests_the_scan_state() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (_before_manager, before) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let mut kept = scan_state_snapshot(&before, &alice.id(), true).await; + kept.capabilities = PersistenceCapabilities::NONE; + let manager = PlatformWalletManager::new( + Arc::new(dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await), + Arc::new(kept), + Arc::new(NoopEventHandler), + ); + manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + let restarted = manager + .get_wallet(&before.wallet_id()) + .await + .expect("the relaunched wallet"); + assert!(restarted.private_payments_held().await, "held"); + managed(&restarted, ALICE, |managed| { + assert_eq!( + managed.dashpay().private_contact_requests_position(), + 0, + "read again from the start" + ); + }) + .await; + } + + /// A private payment is refused before anything is sealed or paid over a + /// backend that does not keep the private contact request scan state, as + /// its recovery marker could not survive a restart; nothing stays + /// reserved. + #[tokio::test] + async fn should_refuse_a_private_payment_over_a_backend_that_keeps_no_marker() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet_on( + RecordingPersistence::attesting(PersistenceCapabilities::NONE), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let refused = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!( + refused, + Err(PlatformWalletError::PrivateContactRecoveryUnsupported) + ), + "{refused:?}" + ); + assert!(!pending_to(&alice_wallet, ALICE, &bob).await); + } + + /// Over a backend that buffers its stores, a private payment's marker is + /// flushed before it is paid: a flush that fails refuses the payment, with + /// nothing paid, and one that succeeds leaves the marker durable before + /// `pay` runs, so a process reopening the wallet, in a build without the + /// `shielded` feature too, holds the identity. + #[tokio::test] + async fn should_make_a_private_payment_s_marker_durable_before_paying() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + + let failing = Arc::new(BufferedPersistence::default()); + let (_manager, alice_wallet) = wallet_on( + Arc::clone(&failing), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + failing.flush_fails.store(true, Ordering::SeqCst); + let refused = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!(&refused, Err(PlatformWalletError::Persistence(message)) + if message.contains("not stored durably")), + "{refused:?}" + ); + assert!(!pending_to(&alice_wallet, ALICE, &bob).await); + + let buffered = Arc::new(BufferedPersistence::default()); + let (_manager, alice_wallet) = wallet_on( + Arc::clone(&buffered), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let durable_when_paid = Arc::new(AtomicBool::new(false)); + let unconfirmed = { + let (buffered, durable, alice_id) = ( + Arc::clone(&buffered), + Arc::clone(&durable_when_paid), + alice.id(), + ); + move |_: BinaryData, started: PaymentStarted| { + let marker = last_marker(&buffered.durable.lock().expect("durable"), &alice_id); + durable.store(marker == Some(true), Ordering::SeqCst); + started.mark(); + std::future::ready(Err::( + PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create", + reason: "the process ends here".to_string(), + }, + )) + } + }; + let sent = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + unconfirmed, + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{sent:?}" + ); + assert!( + durable_when_paid.load(Ordering::SeqCst), + "durable before paying" + ); + + // The process ends with only what was flushed: reopened from it. + let unresolved = + last_marker(&buffered.durable.lock().expect("durable"), &alice.id()) == Some(true); + let kept = Arc::new(scan_state_snapshot(&alice_wallet, &alice.id(), unresolved).await); + let manager = PlatformWalletManager::new( + Arc::new(dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await), + kept, + Arc::new(NoopEventHandler), + ); + manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + let reopened = manager + .get_wallet(&alice_wallet.wallet_id()) + .await + .expect("the relaunched wallet"); + assert!(reopened.private_payments_held().await, "held"); + let public = reopened + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{public:?}" + ); + } + + /// A private payment whose broadcast may have been delivered, an + /// `Unavailable` answer after the node may have admitted it, keeps its + /// pair held and its marker set, though its notes are released: its entry + /// may still be appended. A public send to that recipient is refused. + #[tokio::test] + async fn should_keep_the_pair_held_when_a_payment_may_have_been_delivered() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let persistence = RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + ); + let (_manager, alice_wallet) = wallet_on( + Arc::clone(&persistence), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let view = alice_wallet.identity().dashpay(); + let delivered = |_: BinaryData, started: PaymentStarted| { + started.mark(); + std::future::ready(Err::( + PlatformWalletError::ShieldedBroadcastMaybeDelivered( + "status: Unavailable".to_string(), + ), + )) + }; + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + delivered, + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::ShieldedBroadcastMaybeDelivered(_)) + ), + "{sent:?}" + ); + assert!(pending_to(&alice_wallet, ALICE, &bob).await, "held"); + assert_eq!( + last_marker(&persistence.stored.lock().expect("stored"), &alice.id()), + Some(true), + "marked" + ); + let public = view + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob.id() + ), + "{public:?}" + ); + } + + /// `identity` with each of its `ECDSA_SECP256K1` keys holding its point + /// uncompressed, a 65-byte encoding DPP accepts and the sealer reads. + fn with_uncompressed_keys(identity: &Identity) -> Identity { + Identity::V0(IdentityV0 { + id: identity.id(), + public_keys: identity + .public_keys() + .iter() + .map(|(id, key)| { + let mut key = key.clone(); + if key.key_type() == KeyType::ECDSA_SECP256K1 { + let point = + PublicKey::from_slice(key.data().as_slice()).expect("a valid point"); + key.set_data(BinaryData::new(point.serialize_uncompressed().to_vec())); + } + (*id, key) + }) + .collect(), + balance: identity.balance(), + revision: identity.revision(), + }) + } + + /// A request sealed to a recipient key holding its point uncompressed is + /// recovered from the store by its sender restored from seed, as one sealed + /// to a compressed key is: the key is matched as a curve point. A public + /// send to that recipient is then refused, the requests having gone + /// privately. + #[tokio::test] + async fn should_recover_a_request_sealed_to_an_uncompressed_recipient_key() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = with_uncompressed_keys(&identity(BOB_PHRASE, BOB)); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {sent:?}" + ); + let entry = kept.lock().expect("kept").remove(0); + + let (_restored_manager, restored) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![entry.to_vec()]).await; + let view = restored.identity().dashpay(); + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the store is read"); + managed(&restored, ALICE, |managed| { + assert!( + managed.dashpay().outgoing_request(&bob.id()).is_some(), + "recovered" + ); + }) + .await; + let public = view + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(recipient)) + if recipient == bob.id() + ), + "{public:?}" + ); + } + + /// A saved private payment whose redrive record does not read back, its + /// association with the request it pays for or its signed transition, or + /// was lost, still holds its pair after a relaunch: the shielded store's + /// private payment journal restores it. Through a coordinator sync pass, a second + /// restoration and another relaunch, a private and a public request to + /// that recipient are refused, while one to anyone else is not held. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_hold_a_pair_from_its_journal_when_its_saved_payment_does_not_read() { + for damage in [ + "UPDATE shielded_pending_spends SET private_contact_payment = x'01'", + "UPDATE shielded_pending_spends SET st_bytes = x'0102'", + // The redrive record lost after the broadcast. + "DELETE FROM shielded_pending_spends", + ] { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let incoming = public_request(&alice, &bob, 3); + let dir = tempfile::tempdir().expect("temporary shielded store"); + let db_path = dir.path().join("tree.sqlite"); + + let store = FileBackedShieldedStore::open_path(&db_path, 100).expect("open the store"); + let before_coordinator = Arc::new(NetworkShieldedCoordinator::new( + Arc::new(dash_sdk::Sdk::new_mock()), + Network::Testnet, + db_path.clone(), + store, + )); + let (_before_manager, before) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + add_incoming(&before, BOB, &incoming).await; + // A backend keeping the viewing key and the private scan state, so a + // private send reaches the pair's hold. + let mut kept = snapshot(&before, &bob.id(), BOB_PHRASE).await; + kept.capabilities = kept + .capabilities + .union(PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART); + let kept = Arc::new(kept); + let subwallet = SubwalletId::new(before.wallet_id(), 0); + let pay = { + let store = Arc::clone(before_coordinator.store()); + let (sender, recipient) = (bob.id(), alice.id()); + move |entry: BinaryData, started: PaymentStarted| async move { + let transition = private_contact_request_payment(entry); + let payment = PrivateContactPayment { + sender: sender.to_buffer(), + recipient: recipient.to_buffer(), + account_reference: started + .account_reference() + .expect("the send names its request"), + }; + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x77; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "private document create from shielded pool", + Some(payment), + Some(&started), + 0, + ) + .await?; + Err::(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create", + reason: "the result wait timed out".to_string(), + }) + } + }; + let paid = before + .identity() + .dashpay() + .accept_private_contact_request(&incoming, None, &bob_signer, &bob_crypto, pay) + .await; + assert!( + matches!( + paid, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{paid:?}" + ); + drop(before_coordinator); + rusqlite::Connection::open(&db_path) + .expect("the store's file") + .execute(damage, []) + .expect("the saved payment is damaged"); + + for relaunch in 0..2 { + let restarted_manager = PlatformWalletManager::new( + Arc::new( + dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await, + ), + Arc::clone(&kept), + Arc::new(NoopEventHandler), + ); + restarted_manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + restarted_manager + .configure_shielded(&db_path) + .await + .expect("the shielded store opens"); + let restarted = restarted_manager + .get_wallet(&before.wallet_id()) + .await + .expect("the relaunched wallet"); + let coordinator = restarted_manager + .shielded_coordinator() + .await + .expect("the coordinator"); + for pass in 0..2 { + if pass == 1 { + // A coordinator pass, whose redrive keeps an undecodable + // record, then a second restoration. + coordinator.sync(true, &CancellationToken::new()).await; + restarted + .restore_private_contact_payments(&coordinator) + .await + .expect("restored again from the journal"); + } + let context = format!("{damage}, relaunch {relaunch}, pass {pass}"); + assert!(!restarted.private_payments_held().await, "{context}"); + let view = restarted.identity().dashpay(); + let public = view + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{context}: {public:?}" + ); + let private = view + .send_private_contact_request( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await; + assert!( + matches!( + private, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{context}: {private:?}" + ); + let elsewhere = view + .send_contact_request_with_external_signer( + &bob.id(), + &Identifier::from([0xCC; 32]), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + !matches!( + elsewhere, + Err(PlatformWalletError::PrivateContactRequestPending(_) + | PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{context}: {elsewhere:?}" + ); + } + } + } + } + + /// The page of DashPay's private store from `start`, holding `entries`. + fn store_page(start: u64, entries: Vec>) -> PrivateDocumentEntries { + let total_count = start + entries.len() as u64; + PrivateDocumentEntries { + chunk_power: None, + entries: entries + .into_iter() + .zip(start..) + .map(|(data, position)| PrivateDocumentEntry { + position, + block_height: 0, + block_time_ms: BLOCK_TIME_MS, + data, + }) + .collect(), + total_count, + } + } + + /// From now on, `wallet`'s mock SDK answers the page from `start` with + /// `entries`: the store as it is once they are appended. + async fn publish(wallet: &PlatformWallet, start: u64, entries: Vec>) { + let contract = dashpay_contract().expect("bundled DashPay"); + let query = PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + start, + ); + let mut sdk = (*wallet.identity().sdk).clone(); + let mut mock = sdk.mock(); + mock.remove_fetch_expectation::(query.clone()) + .await; + mock.expect_fetch(query, Some(store_page(start, entries))) + .await + .expect("page expectation"); + } + + /// A `pay` that appends the entry to the store `wallet` reads, after + /// `before` from `start`, and keeps it. + fn append_entry( + wallet: &Arc, + start: u64, + before: Vec>, + kept: &Arc>>, + ) -> impl FnOnce( + BinaryData, + PaymentStarted, + ) -> std::pin::Pin< + Box> + Send>, + > + Send { + let wallet = Arc::clone(wallet); + let kept = Arc::clone(kept); + move |entry, started| { + started.mark(); + Box::pin(async move { + let mut entries = before; + entries.push(entry.to_vec()); + publish(&wallet, start, entries).await; + kept.lock().expect("kept entries").push(entry); + Ok(ResponseMetadata { + time_ms: BLOCK_TIME_MS, + ..Default::default() + }) + }) + } + } + + /// A `pay` that keeps the entry instead of paying for it, and answers as + /// the proof of a block at `BLOCK_TIME_MS` would: the entry never reaches + /// the store. + fn keep_entry( + kept: &Arc>>, + ) -> impl FnOnce( + BinaryData, + PaymentStarted, + ) -> std::future::Ready> + + Send { + let kept = Arc::clone(kept); + move |entry, started| { + started.mark(); + kept.lock().expect("kept entries").push(entry); + std::future::ready(Ok(ResponseMetadata { + time_ms: BLOCK_TIME_MS, + ..Default::default() + })) + } + } + + async fn managed( + wallet: &PlatformWallet, + identity_id: [u8; 32], + read: impl FnOnce(&crate::wallet::identity::ManagedIdentity) -> R, + ) -> R { + let iw = wallet.identity(); + let wm = iw.wallet_manager.read().await; + let managed = wm + .get_wallet_info(&wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity(&Identifier::from(identity_id)) + .expect("managed identity"); + read(managed) + } + + /// Alice sends Bob a private request; Bob reads it from the store with + /// Alice checked as its sender, refuses to accept it with a public reply, + /// and accepts it with a private one, which establishes the contact. The + /// request Bob records carries Alice's receiving xpub and label as a + /// document would, under the pair's shared key. + #[tokio::test] + async fn should_send_read_and_accept_a_private_contact_request() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let bob_signer = signer(BOB_PHRASE, &bob); + let shielded_address = [0x55; 43]; + + let (_alice_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + Some("Alice".to_string()), + AutoAcceptProofSource::None, + Some(shielded_address), + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the request, read back from the store"); + let entry = kept.lock().expect("kept entries").remove(0); + assert_eq!(entry.len(), 589, "the entry takes DashPay's entry size"); + assert_eq!(sent.sender_key_index, ENCRYPTION_KEY_ID); + assert_eq!(sent.recipient_key_index, DECRYPTION_KEY_ID); + assert_eq!(sent.created_at, BLOCK_TIME_MS); + assert!(sent.came_privately(), "recorded at core height 0"); + managed(&alice_wallet, ALICE, |managed| { + assert_eq!( + managed.dashpay().sent_contact_requests().get(&bob.id()), + Some(&sent) + ); + assert!( + managed + .dashpay() + .private_contact(&bob.id()) + .expect("the request went privately") + .sent_privately + ); + }) + .await; + // Alice's public request to Bob would publish what she sent privately. + assert!(matches!( + alice_wallet + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(recipient)) + if recipient == bob.id() + )); + + let (_bob_manager, bob_wallet) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry.to_vec()]).await; + let bob_view = bob_wallet.identity().dashpay(); + let report = bob_view + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + assert_eq!(report.pages_read, 1); + assert!(report.complete); + assert_eq!(report.next_position, 1); + assert_eq!(report.rejected, vec![]); + let received = managed(&bob_wallet, BOB, |managed| { + assert_eq!(managed.dashpay().private_contact_requests_position(), 1); + let private = managed + .dashpay() + .private_contact(&alice.id()) + .expect("noted"); + assert!(private.received_privately); + assert_eq!(private.shielded_address, Some(shielded_address)); + managed + .dashpay() + .incoming_contact_requests() + .get(&alice.id()) + .cloned() + .expect("Bob holds Alice's request") + }) + .await; + assert_eq!(received.sender_key_index, ENCRYPTION_KEY_ID); + assert_eq!(received.recipient_key_index, DECRYPTION_KEY_ID); + assert_eq!(received.account_reference, sent.account_reference); + assert_eq!(received.created_at, BLOCK_TIME_MS); + assert!(received.came_privately(), "recorded at core height 0"); + assert_eq!(received.auto_accept_proof, None); + // The shared key Bob's decryption key and Alice's encryption key agree + // on opens the xpub and label, as it opens a document's. + let bob_key_path = identity_auth_derivation_path_for_type( + Network::Testnet, + KeyDerivationType::ECDSA, + 0, + DECRYPTION_KEY_ID, + ) + .expect("Bob's decryption key path"); + let shared_key = bob_crypto + .ecdh_shared_secret( + &bob_key_path, + &identity_public_key(&alice, ENCRYPTION_KEY_ID).expect("Alice's key"), + ) + .await + .expect("shared key"); + assert_eq!( + decrypt_extended_public_key(&shared_key, &received.encrypted_public_key) + .expect("the xpub opens"), + decrypt_extended_public_key(&shared_key, &sent.encrypted_public_key) + .expect("Alice's record opens with the same key"), + ); + assert_eq!( + received.encrypted_public_key, sent.encrypted_public_key, + "both sides record the same ciphertext" + ); + assert_eq!( + decrypt_account_label( + &shared_key, + received + .encrypted_account_label + .as_deref() + .expect("a label") + ) + .expect("the label opens"), + "Alice" + ); + + // A public reply would publish the relationship. + assert!(matches!( + bob_view + .accept_contact_request_with_external_signer(&received, &bob_signer, &bob_crypto) + .await, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(sender)) + if sender == alice.id() + )); + + // The accept a host calls replies the way the request came. + let reply = Arc::new(Mutex::new(Vec::new())); + let established = bob_view + .accept_contact_request_replying_in_kind( + &received, + None, + &bob_signer, + &bob_crypto, + append_entry(&bob_wallet, 1, vec![], &reply), + ) + .await + .expect("Bob accepts privately"); + assert_eq!(established.contact_identity_id, alice.id()); + assert_eq!(established.incoming_request, received); + assert_eq!( + reply.lock().expect("reply").len(), + 1, + "the reply is a private entry" + ); + managed(&bob_wallet, BOB, |managed| { + let private = managed + .dashpay() + .private_contact(&alice.id()) + .expect("noted"); + assert!(private.received_privately && private.sent_privately); + }) + .await; + // Bob can pay Alice: her sending account is registered from the xpub. + { + let iw = bob_wallet.identity(); + let wm = iw.wallet_manager.read().await; + let info = wm + .get_wallet_info(&bob_wallet.wallet_id()) + .expect("wallet info"); + assert!(info + .core_wallet + .accounts + .dashpay_external_accounts + .contains_key(&DashpayAccountKey { + index: 0, + user_identity_id: BOB, + friend_identity_id: ALICE, + })); + } + + // Un-ignoring a sender reads the store again from its start, where the + // requests skipped while it was ignored are. + { + let iw = bob_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&bob_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob"); + managed.ignore_sender(&alice.id()); + // Past Alice's request and Bob's reply, which the accept read back. + assert_eq!(managed.dashpay().private_contact_requests_position(), 2); + managed.unignore_sender(&alice.id()); + assert_eq!(managed.dashpay().private_contact_requests_position(), 0); + } + } + + /// A public request from `sender` to `recipient` for `account_reference`, + /// as a `contactRequest` document is recorded: with its core height. + fn public_request( + sender: &Identity, + recipient: &Identity, + account_reference: u32, + ) -> ContactRequest { + ContactRequest::new( + sender.id(), + recipient.id(), + ENCRYPTION_KEY_ID, + DECRYPTION_KEY_ID, + account_reference, + vec![7; 96], + 100, + BLOCK_TIME_MS, + ) + } + + /// The accept a host calls replies to a public request publicly, without + /// paying: by adopting the reply already sent, or with a document whose + /// broadcast error comes back as it is. A request the wallet does not + /// hold is refused either way, and nothing is paid. + #[tokio::test] + async fn should_accept_a_public_request_replying_in_kind() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let incoming = public_request(&alice, &bob, 3); + let add = |wallet: Arc, requests: Vec<(bool, ContactRequest)>| async move { + let iw = wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&Identifier::from(BOB)) + .expect("Bob"); + for (incoming, request) in requests { + if incoming { + managed + .add_incoming_contact_request(request, &iw.persister) + .expect("incoming request"); + } else { + managed + .add_sent_contact_request(request, &iw.persister) + .expect("sent request"); + } + } + }; + + // Already reciprocated: adopted, no broadcast and no payment. + let (_manager, reciprocated) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + add( + Arc::clone(&reciprocated), + vec![ + (true, incoming.clone()), + (false, public_request(&bob, &alice, 4)), + ], + ) + .await; + let paid = Arc::new(Mutex::new(Vec::new())); + let established = reciprocated + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &bob_signer, + &bob_crypto, + keep_entry(&paid), + ) + .await + .expect("the reply already sent is adopted"); + assert_eq!(established.contact_identity_id, alice.id()); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + + // Not reciprocated: a public reply, whose broadcast this mock refuses. + let (_manager, fresh) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + add(Arc::clone(&fresh), vec![(true, incoming.clone())]).await; + let paid = Arc::new(Mutex::new(Vec::new())); + let result = fresh + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &bob_signer, + &bob_crypto, + keep_entry(&paid), + ) + .await; + assert!( + matches!(&result, Err(error) + if !matches!(error, PlatformWalletError::PrivateContactRequestNeedsPrivateReply(_))), + "the public reply's error comes back: {result:?}" + ); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + + // A private request from a sender the wallet holds no request from. + let carol = identity(ALICE_PHRASE, [0xC1; 32]); + let mut unknown = public_request(&carol, &bob, 5); + unknown.core_height_created_at = 0; + let paid = Arc::new(Mutex::new(Vec::new())); + assert!(matches!( + fresh + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &unknown, + None, + &bob_signer, + &bob_crypto, + keep_entry(&paid), + ) + .await, + Err(PlatformWalletError::ContactRequestNotFound(sender)) if sender == carol.id() + )); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + } + + /// A wallet restored from seed recovers the private requests it sent from + /// the store alone, recording each as it recorded it when sending. + #[tokio::test] + async fn should_recover_a_sent_private_contact_request_from_the_store() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the request"); + let entry = kept.lock().expect("kept entries").remove(0); + + // The restored wallet checks its own identity's keys as on chain. + let (_restored_manager, restored) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![entry.to_vec()]).await; + let report = restored + .identity() + .dashpay() + .sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the restored wallet reads the store"); + assert_eq!(report.requests, vec![sent.clone()]); + managed(&restored, ALICE, |managed| { + assert_eq!( + managed.dashpay().sent_contact_requests().get(&bob.id()), + Some(&sent) + ); + }) + .await; + } + + /// A sync reports the requests it recorded: an ignored sender's request is + /// opened but not recorded, so not reported; once the sender is no longer + /// ignored, the store is read again and the request is. + #[tokio::test] + async fn should_report_only_the_requests_a_sync_recorded() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the request"); + let entry = kept.lock().expect("kept entries").remove(0); + + let (_bob_manager, bob_wallet) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry.to_vec()]).await; + let set_ignored = |ignored: bool| { + let bob_wallet = Arc::clone(&bob_wallet); + let alice_id = alice.id(); + let bob_id = bob.id(); + async move { + let iw = bob_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&bob_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob_id) + .expect("Bob"); + if ignored { + managed.ignore_sender(&alice_id); + } else { + managed.unignore_sender(&alice_id); + } + } + }; + + set_ignored(true).await; + let report = bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + assert_eq!(report.requests, vec![], "an ignored sender's request"); + assert_eq!(report.rejected, vec![]); + + set_ignored(false).await; + let report = bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store again"); + assert_eq!(report.requests.len(), 1, "{:?}", report.requests); + assert_eq!(report.requests[0].sender_id, alice.id()); + managed(&bob_wallet, BOB, |managed| { + assert_eq!( + managed.dashpay().incoming_request(&alice.id()), + Some(&report.requests[0]) + ); + }) + .await; + } + + /// One private request to a recipient is in flight at a time: one paid + /// for but not read back, or whose payment is unconfirmed, keeps the next + /// from being sent, until a sync reads it back; one whose payment failed + /// does not. + #[tokio::test] + async fn should_send_one_private_request_to_a_recipient_at_a_time() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + macro_rules! send { + ($pay:expr) => { + view.send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + $pay, + ) + .await + }; + } + let pending = |result: &Result| matches!(result, Err(PlatformWalletError::PrivateContactRequestPending(id)) if *id == bob.id()); + + // A payment that definitely failed releases the recipient. + let failed = send!(|_, _| std::future::ready(Err( + PlatformWalletError::InvalidIdentityData("no notes".to_string()) + ))); + assert!(!pending(&failed), "{failed:?}"); + + // Paid for, but the entry is not in the store yet: pending. + let kept = Arc::new(Mutex::new(Vec::new())); + let first = send!(keep_entry(&kept)); + assert!(pending(&first), "{first:?}"); + let entry = kept.lock().expect("kept").remove(0); + + // A second request is refused before anything is paid. + let paid_again = Arc::new(Mutex::new(Vec::new())); + let second = send!(keep_entry(&paid_again)); + assert!(pending(&second), "{second:?}"); + assert!( + paid_again.lock().expect("paid").is_empty(), + "nothing is paid twice" + ); + + // The store reads the entry back: settled and recorded. + publish(&alice_wallet, 0, vec![entry.to_vec()]).await; + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("Alice reads the store"); + managed(&alice_wallet, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + assert!(managed.dashpay().outgoing_request(&bob.id()).is_some()); + }) + .await; + } + + /// A payment whose execution is unconfirmed keeps the request pending. + #[tokio::test] + async fn should_keep_an_unconfirmed_private_request_pending() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let result = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + |_, started: PaymentStarted| { + started.mark(); + std::future::ready(Err(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create", + reason: "the proof fetch timed out".to_string(), + })) + }, + ) + .await; + assert!(matches!( + result, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + )); + managed(&alice_wallet, ALICE, |managed| { + assert!(managed.dashpay().private_reply_pending(&bob.id())); + }) + .await; + // A public request now would publish the pair, whose private one may + // already be on chain. + assert!(matches!( + alice_wallet + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await, + Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob.id() + )); + } + + /// A signer whose signatures never come, as a host's prompt the user never + /// answers: it notes that it was asked. + #[derive(Debug)] + struct StalledSigner { + inner: simple_signer::signer::SimpleSigner, + asked: Arc, + } + + #[async_trait::async_trait] + impl Signer for StalledSigner { + async fn sign( + &self, + _key: &IdentityPublicKey, + _data: &[u8], + ) -> Result { + self.asked.store(true, Ordering::SeqCst); + std::future::pending().await + } + + async fn sign_create_witness( + &self, + key: &IdentityPublicKey, + data: &[u8], + ) -> Result { + self.inner.sign_create_witness(key, data).await + } + + fn can_sign_with(&self, key: &IdentityPublicKey) -> bool { + self.inner.can_sign_with(key) + } + } + + /// From now on, `wallet`'s mock SDK answers a fetch of `identity` with it + /// when `known`, and fails it otherwise. + async fn set_identity_known(wallet: &PlatformWallet, identity: &Identity, known: bool) { + let mut sdk = (*wallet.identity().sdk).clone(); + let mut mock = sdk.mock(); + mock.remove_fetch_expectation::(identity.id()) + .await; + if known { + mock.expect_fetch(identity.id(), Some(identity.clone())) + .await + .expect("identity expectation"); + } + } + + /// Whether `wallet`'s identity `identity_id` holds a private request to + /// `recipient` pending. + async fn pending_to( + wallet: &PlatformWallet, + identity_id: [u8; 32], + recipient: &Identity, + ) -> bool { + managed(wallet, identity_id, |managed| { + managed.dashpay().private_reply_pending(&recipient.id()) + }) + .await + } + + /// A public request is bound to its wallet's generation: while one is on + /// its way, paused here at its signature after claiming the pair, the + /// wallet cannot be removed; once the wallet is removed and re-created + /// under the same id, a send through the old handle is refused before it + /// decides anything, publishing nothing and changing nothing on the + /// replacement. + #[tokio::test] + async fn should_not_let_a_public_request_outlive_its_wallet() { + use std::future::Future; + use std::task::{Context, Waker}; + + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (manager, old) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let wallet_id = old.wallet_id(); + let (alice_id, bob_id) = (alice.id(), bob.id()); + { + // Paused right after it takes the wallet's lifecycle gate, at its + // check that the wallet is still live. + let held = old.wallet_manager().write().await; + let view = old.identity().dashpay(); + let mut send = Box::pin(view.send_contact_request_with_external_signer( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + )); + let mut cx = Context::from_waker(Waker::noop()); + assert!(send.as_mut().poll(&mut cx).is_pending()); + // In flight, it holds the wallet's lifecycle gate, which a removal + // takes exclusively before anything else: no removal can complete. + assert!( + tokio::time::timeout( + Duration::from_millis(100), + old.generation().teardown_guard() + ) + .await + .is_err(), + "a removal waits for the public request" + ); + // Dropped, the send gives the wallet up. + drop(send); + drop(held); + } + + manager.remove_wallet(&wallet_id).await.expect("removed"); + let replacement = manager + .create_wallet_from_seed_bytes( + Network::Testnet, + &seed(ALICE_PHRASE), + WalletAccountCreationOptions::Default, + Some(0), + ) + .await + .expect("re-created"); + assert_eq!(replacement.wallet_id(), wallet_id); + { + let iw = replacement.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&wallet_id) + .expect("the replacement's info") + .identity_manager + .add_identity(alice.clone(), 0, wallet_id, &iw.persister) + .expect("alice"); + } + let resumed = old + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &signer(ALICE_PHRASE, &alice), + &alice_crypto, + ) + .await; + assert!( + matches!( + resumed, + Err(PlatformWalletError::WalletRemovedBeforeSend(_)) + ), + "{resumed:?}" + ); + managed(&replacement, ALICE, |managed| { + assert!(managed.dashpay().outgoing_request(&bob.id()).is_none()); + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + }) + .await; + } + + /// A send dropped before it pays, here while its signer never answers, + /// leaves no reservation behind: nothing was paid, and no sync could + /// settle one. The recipient is free for the next send. + #[tokio::test] + async fn should_release_the_reservation_of_a_send_dropped_before_it_pays() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + let asked = Arc::new(AtomicBool::new(false)); + let stalled = StalledSigner { + inner: signer(ALICE_PHRASE, &alice), + asked: Arc::clone(&asked), + }; + let paid = Arc::new(Mutex::new(Vec::new())); + + let cancelled = tokio::time::timeout( + Duration::from_millis(200), + view.send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &stalled, + &alice_crypto, + keep_entry(&paid), + ), + ) + .await; + assert!(cancelled.is_err(), "the send never completes"); + assert!( + asked.load(Ordering::SeqCst), + "it was dropped while signing, so after reserving" + ); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + assert!(!pending_to(&alice_wallet, ALICE, &bob).await); + + let kept = Arc::new(Mutex::new(Vec::new())); + let next = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &signer(ALICE_PHRASE, &alice), + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + next, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{next:?}" + ); + assert_eq!(kept.lock().expect("kept").len(), 1, "the next send pays"); + } + + /// A sync settles a pending request only once it records the entry: one + /// whose recipient cannot be read leaves no record, so the request stays + /// pending, and it settles on a later sync that reads the recipient. + #[tokio::test] + async fn should_settle_a_pending_request_only_once_its_entry_is_recorded() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{sent:?}" + ); + let entry = kept.lock().expect("kept").remove(0); + publish(&alice_wallet, 0, vec![entry.to_vec()]).await; + + set_identity_known(&alice_wallet, &bob, false).await; + let report = view + .sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("Alice reads the store"); + assert_eq!(report.requests, vec![]); + assert_eq!(report.unpersisted_identities, vec![alice.id()]); + managed(&alice_wallet, ALICE, |managed| { + assert!(managed.dashpay().private_reply_pending(&bob.id())); + assert!(managed.dashpay().outgoing_request(&bob.id()).is_none()); + }) + .await; + + set_identity_known(&alice_wallet, &bob, true).await; + let report = view + .sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("Alice reads the store again"); + assert_eq!(report.requests.len(), 1, "{:?}", report.requests); + managed(&alice_wallet, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + assert_eq!( + managed.dashpay().outgoing_request(&bob.id()), + Some(&report.requests[0]) + ); + }) + .await; + } + + /// A pending request whose entry is read back after a newer request to + /// the recipient is tracked, here as another device's rotation would be, + /// is settled: its entry executed, and the newer request stays tracked. + #[tokio::test] + async fn should_settle_a_pending_request_read_back_after_a_newer_one() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{sent:?}" + ); + let entry = kept.lock().expect("kept").remove(0); + + let mut newer = public_request(&alice, &bob, (1 << 28) | 5); + newer.core_height_created_at = 0; + { + let iw = alice_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&alice_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("Alice") + .add_sent_contact_request(newer.clone(), &iw.persister) + .expect("the newer request"); + } + publish(&alice_wallet, 0, vec![entry.to_vec()]).await; + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("Alice reads the store"); + managed(&alice_wallet, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + assert_eq!(managed.dashpay().outgoing_request(&bob.id()), Some(&newer)); + }) + .await; + } + + /// A request paid for and unconfirmed before a restart is restored from + /// its entry: until a sync reads the entry back, neither a second private + /// request nor a public one goes to the recipient. Once recorded, the + /// entry restores nothing. + #[tokio::test] + async fn should_restore_a_pending_request_paid_for_before_a_restart() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, before) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + before + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await + .expect_err("not read back"); + let entry = kept.lock().expect("kept").remove(0); + + let (_restarted_manager, restarted) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = restarted.identity().dashpay(); + assert!(!pending_to(&restarted, ALICE, &bob).await); + let restored = view + .restore_pending_private_contact_requests(std::slice::from_ref(&entry), &alice_crypto) + .await + .expect("restored"); + assert_eq!(restored, vec![(alice.id(), bob.id())]); + assert!(pending_to(&restarted, ALICE, &bob).await); + + let paid = Arc::new(Mutex::new(Vec::new())); + let second = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&paid), + ) + .await; + assert!( + matches!(second, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob.id()), + "{second:?}" + ); + assert!( + paid.lock().expect("paid").is_empty(), + "nothing is paid twice" + ); + let public = view + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!(public, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob.id()), + "{public:?}" + ); + + publish(&restarted, 0, vec![entry.to_vec()]).await; + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the restarted wallet reads the store"); + managed(&restarted, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + assert!(managed.dashpay().outgoing_request(&bob.id()).is_some()); + }) + .await; + assert_eq!( + view.restore_pending_private_contact_requests(&[entry], &alice_crypto) + .await + .expect("restored"), + vec![], + "a recorded entry is not restored" + ); + } + + /// A send whose payment finds a request to the same recipient paid for + /// and unconfirmed before a restart pays nothing: the recipient stays + /// reserved for that request, which a sync settles once it reads it back. + #[tokio::test] + async fn should_hold_a_send_for_a_request_paid_for_before_a_restart() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (alice_id, bob_id) = (alice.id(), bob.id()); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, before) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + before + .identity() + .dashpay() + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await + .expect_err("not read back"); + let unconfirmed = vec![kept.lock().expect("kept").remove(0)]; + + // The host restores nothing at startup; the payment checks. + let (_restarted_manager, restarted) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = restarted.identity().dashpay(); + let paid = Arc::new(Mutex::new(Vec::new())); + let held = view + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + |entry: BinaryData, started: PaymentStarted| { + let paid = Arc::clone(&paid); + let unconfirmed = &unconfirmed; + let crypto = &alice_crypto; + async move { + if view + .hold_for_restored_private_contact_request( + unconfirmed, + crypto, + &alice_id, + &bob_id, + ) + .await? + { + return Err(PlatformWalletError::PrivateContactRequestPending(bob_id)); + } + started.mark(); + paid.lock().expect("paid").push(entry); + Ok(ResponseMetadata::default()) + } + }, + ) + .await; + assert!( + matches!(held, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob_id), + "{held:?}" + ); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + assert!(pending_to(&restarted, ALICE, &bob).await); + + // The restored entry, not the unpaid one, settles it. + publish(&restarted, 0, vec![unconfirmed[0].to_vec()]).await; + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the restarted wallet reads the store"); + assert!(!pending_to(&restarted, ALICE, &bob).await); + } + + /// A rotation whose payment is unconfirmed at a restart, over a request + /// that was recorded, is restored: the recorded request is older, so it + /// does not discharge the rotation. Until the rotation is read back, a + /// retried rotation is refused unpaid, whether the host restored at + /// startup or the payment finds it; once it is recorded, its entry, and + /// the older one, restore nothing. + #[tokio::test] + async fn should_restore_a_rotation_paid_for_before_a_restart() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (alice_id, bob_id) = (alice.id(), bob.id()); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, before) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + let v0 = before + .identity() + .dashpay() + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&before, 0, vec![], &kept), + ) + .await + .expect("v0 is recorded"); + assert_eq!(account_reference_version(v0.account_reference), 0); + let v0_entry = kept.lock().expect("kept").remove(0); + let unconfirmed = { + let kept = Arc::clone(&kept); + move |entry: BinaryData, started: PaymentStarted| { + started.mark(); + kept.lock().expect("kept").push(entry); + std::future::ready(Err::( + PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create", + reason: "the proof fetch timed out".to_string(), + }, + )) + } + }; + let rotation = before + .identity() + .dashpay() + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + unconfirmed, + ) + .await; + assert!( + matches!( + rotation, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{rotation:?}" + ); + let v1_entry = kept.lock().expect("kept").remove(0); + + // A restart: the store holds v0 only, which the restarted wallet + // records. The host restores the unconfirmed rotation. + let (_restarted_manager, restarted) = wallet( + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![v0_entry.to_vec()], + ) + .await; + let view = restarted.identity().dashpay(); + view.sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the restarted wallet reads the store"); + managed(&restarted, ALICE, |managed| { + assert_eq!(managed.dashpay().outgoing_request(&bob_id), Some(&v0)); + }) + .await; + assert_eq!( + view.restore_pending_private_contact_requests( + std::slice::from_ref(&v1_entry), + &alice_crypto + ) + .await + .expect("restored"), + vec![(alice_id, bob_id)], + "a recorded v0 does not discharge the v1 rotation" + ); + let retried = view + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!(retried, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob_id), + "{retried:?}" + ); + + // Another restart, the host restoring nothing: the payment of the + // retried rotation finds the unconfirmed one and pays nothing. + let (_held_manager, held_wallet) = wallet( + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![v0_entry.to_vec()], + ) + .await; + let held_view = held_wallet.identity().dashpay(); + let paid = Arc::new(Mutex::new(Vec::new())); + let unconfirmed_entries = vec![v1_entry.clone()]; + let held = held_view + .send_private_contact_request( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + |entry: BinaryData, started: PaymentStarted| { + let paid = Arc::clone(&paid); + let unconfirmed_entries = &unconfirmed_entries; + let crypto = &alice_crypto; + async move { + if held_view + .hold_for_restored_private_contact_request( + unconfirmed_entries, + crypto, + &alice_id, + &bob_id, + ) + .await? + { + return Err(PlatformWalletError::PrivateContactRequestPending(bob_id)); + } + started.mark(); + paid.lock().expect("paid").push(entry); + Ok(ResponseMetadata::default()) + } + }, + ) + .await; + assert!( + matches!(held, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob_id), + "{held:?}" + ); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + + // v1 lands and is read back, after v0 where each wallet's cursor is: + // settled, and discharged. + for wallet in [&restarted, &held_wallet] { + publish(wallet, 1, vec![v1_entry.to_vec()]).await; + wallet + .identity() + .dashpay() + .sync_private_contact_requests(&alice_crypto, 4) + .await + .expect("the store is read"); + managed(wallet, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob_id)); + let tracked = managed + .dashpay() + .outgoing_request(&bob_id) + .expect("tracked"); + assert_eq!(account_reference_version(tracked.account_reference), 1); + }) + .await; + } + assert_eq!( + view.restore_pending_private_contact_requests(&[v1_entry, v0_entry], &alice_crypto) + .await + .expect("restored"), + vec![], + "a recorded rotation, and the request it rotated, restore nothing" + ); + } + + /// From now on, `wallet`'s mock SDK serves a store holding `entries` in + /// chunks of `chunk_size`, a page from each chunk's start to its end or + /// the store's. + async fn publish_chunked(wallet: &PlatformWallet, entries: &[Vec], chunk_size: usize) { + let total = entries.len() as u64; + for (index, chunk) in entries.chunks(chunk_size).enumerate() { + let start = (index * chunk_size) as u64; + let mut page = store_page(start, chunk.to_vec()); + page.total_count = total; + let contract = dashpay_contract().expect("bundled DashPay"); + let query = PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + start, + ); + let mut sdk = (*wallet.identity().sdk).clone(); + let mut mock = sdk.mock(); + mock.remove_fetch_expectation::(query.clone()) + .await; + mock.expect_fetch(query, Some(page)) + .await + .expect("page expectation"); + } + } + + /// What `wallet` tracks with `contacts` after a sync: each one's requests + /// both ways. + async fn tracked( + wallet: &PlatformWallet, + identity_id: [u8; 32], + contacts: &[Identifier], + ) -> Vec<(Option, Option)> { + managed(wallet, identity_id, |managed| { + contacts + .iter() + .map(|contact| { + ( + managed.dashpay().outgoing_request(contact).cloned(), + managed.dashpay().incoming_request(contact).cloned(), + ) + }) + .collect() + }) + .await + } + + /// A store of many pages is read in one pass, its pages fetched together + /// and scanned in batches: the wallet records what it records from the + /// same entries in one page, and derives the scan keys once a batch, not + /// once a page. + #[tokio::test] + async fn should_sync_a_store_of_many_pages_in_one_pass() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let carol = identity(CAROL_PHRASE, CAROL); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let bob_signer = signer(BOB_PHRASE, &bob); + let everyone = [&alice, &bob, &carol]; + + // Alice's requests to Bob and Carol, and Bob's to Alice. + let (_alice_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &everyone, vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + for (start, recipient) in [(0, bob.id()), (1, carol.id())] { + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &recipient, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, start, vec![], &kept), + ) + .await + .expect("Alice sends"); + } + let to_bob = kept.lock().expect("kept")[0].to_vec(); + let (_bob_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &everyone, vec![to_bob]).await; + bob_wallet + .identity() + .dashpay() + .send_private_contact_request( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + None, + &bob_signer, + &bob_crypto, + append_entry(&bob_wallet, 1, vec![], &kept), + ) + .await + .expect("Bob sends"); + let entries: Vec> = kept + .lock() + .expect("kept") + .iter() + .map(|entry| entry.to_vec()) + .collect(); + assert_eq!(entries.len(), 3); + let contacts = [bob.id(), carol.id()]; + + // Restored from seed, once with the store in one page and once in + // pages of one entry. + let (_one_manager, one_page) = + wallet(ALICE_PHRASE, &alice, &everyone, entries.clone()).await; + let one_page_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let one = one_page + .identity() + .dashpay() + .sync_private_contact_requests(&one_page_crypto, 64) + .await + .expect("the one-page store reads"); + assert_eq!(one.pages_read, 1); + + let (_many_manager, many_pages) = wallet(ALICE_PHRASE, &alice, &everyone, vec![]).await; + publish_chunked(&many_pages, &entries, 1).await; + let many_pages_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let many = many_pages + .identity() + .dashpay() + .sync_private_contact_requests(&many_pages_crypto, 64) + .await + .expect("the store of many pages reads"); + assert_eq!(many.pages_read, 3); + assert!(many.complete); + assert_eq!(many.next_position, 3); + assert_eq!(many.rejected, vec![]); + assert_eq!(many.requests.len(), 3, "{:?}", many.requests); + // A page's received requests are recorded before its sent ones, so + // the order differs; the requests do not. + let sorted = |mut requests: Vec| { + requests.sort_by_key(|request| (request.sender_id, request.recipient_id)); + requests + }; + assert_eq!(sorted(many.requests.clone()), sorted(one.requests.clone())); + assert_eq!( + tracked(&many_pages, ALICE, &contacts).await, + tracked(&one_page, ALICE, &contacts).await + ); + let derivations = many_pages_crypto + .scan_key_derivations + .load(std::sync::atomic::Ordering::SeqCst); + assert!( + (1..many.pages_read).contains(&derivations), + "{derivations} derivations for {} pages", + many.pages_read + ); + } + + /// Alice's private request to Bob, carrying `shielded_address`, read back + /// from the store: its entry and Bob's record of it. + async fn alice_to_bob(shielded_address: [u8; 43]) -> (Vec, ContactRequest) { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let (_alice_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + Some(shielded_address), + &signer(ALICE_PHRASE, &alice), + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends"); + let entry = kept.lock().expect("kept").remove(0).to_vec(); + let (_bob_manager, bob_wallet) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry.clone()]).await; + bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + let received = managed(&bob_wallet, BOB, |managed| { + managed.dashpay().incoming_request(&alice.id()).cloned() + }) + .await + .expect("Bob holds Alice's request"); + (entry, received) + } + + /// For a backend attesting the scan state, a sync stores the private + /// contact's shielded address and then the read position, after the + /// request the entry held. + #[tokio::test] + async fn should_store_the_read_position_after_what_the_entries_held() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let address = [0x55; 43]; + let (entry, _) = alice_to_bob(address).await; + let persistence = RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + ); + let (_manager, bob_wallet) = wallet_on( + Arc::clone(&persistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![entry], + ) + .await; + bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + + let contacts = persistence.contacts(); + let index_of = |found: &dyn Fn(&ContactChangeSet) -> bool| { + contacts + .iter() + .position(found) + .expect("the changeset is stored") + }; + let request = index_of(&|changeset| !changeset.incoming_requests.is_empty()); + let addresses = index_of(&|changeset| { + changeset + .private_shielded_addresses + .get(&(bob.id(), alice.id())) + == Some(&address) + }); + let position = index_of(&|changeset| { + changeset + .private_contact_requests_positions + .get(&bob.id()) + .map(|position| position.position) + == Some(1) + }); + assert!( + request < addresses && addresses < position, + "request {request}, addresses {addresses}, position {position}" + ); + } + + /// A wallet loaded with a stored read position and shielded address, as a + /// backend attesting them restores them, reads the store on from there: + /// no page before it is fetched, which the mock, answering none, would + /// fail. The contact's shielded address is kept. + #[tokio::test] + async fn should_resume_a_private_sync_from_a_restored_position() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let address = [0x55; 43]; + let (entry, received) = alice_to_bob(address).await; + let persistence = RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + ); + let (_manager, restored) = + wallet_on(persistence, BOB_PHRASE, &bob, &[&alice, &bob], vec![entry]).await; + { + let iw = restored.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&restored.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob"); + managed.apply_incoming_contact_request(received.clone()); + let key_fingerprint = managed.private_scan_key_fingerprint(); + managed.apply_private_contact_requests_position(PrivateContactRequestsPosition { + position: 1, + key_fingerprint, + private_payments_unresolved: false, + }); + managed.apply_private_shielded_address(alice.id(), address); + } + let contract = dashpay_contract().expect("bundled DashPay"); + { + let mut sdk = (*restored.identity().sdk).clone(); + sdk.mock() + .remove_fetch_expectation::( + PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + ), + ) + .await; + } + publish(&restored, 1, vec![]).await; + + let report = restored + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("the store reads on from position 1"); + assert_eq!(report.pages_read, 1); + assert!(report.complete); + assert_eq!(report.next_position, 1); + managed(&restored, BOB, |managed| { + assert_eq!( + managed + .dashpay() + .private_contact(&alice.id()) + .expect("a private contact") + .shielded_address, + Some(address) + ); + assert_eq!( + managed.dashpay().incoming_request(&alice.id()), + Some(&received) + ); + }) + .await; + } + + /// Un-ignoring a sender rewinds the private read position, and a backend + /// attesting the scan state, with atomic changesets, stores the rewind in + /// the changeset that stores the sender's removal, to commit together. + #[tokio::test] + async fn should_store_a_rewound_position_when_a_sender_is_unignored() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let persistence = RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART, + ); + let (_manager, bob_wallet) = wallet_on( + Arc::clone(&persistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![], + ) + .await; + let view = bob_wallet.identity().dashpay(); + { + let iw = bob_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&bob_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob") + .apply_private_contact_requests_position(PrivateContactRequestsPosition { + position: 7, + key_fingerprint: [0; 32], + private_payments_unresolved: false, + }); + } + view.ignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("ignored"); + view.unignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("un-ignored"); + let unignore = persistence + .contacts() + .into_iter() + .find(|changeset| changeset.unignored.contains(&(bob.id(), alice.id()))) + .expect("the un-ignore is stored"); + assert_eq!( + unignore + .private_contact_requests_positions + .iter() + .map(|(owner, position)| (*owner, position.position)) + .collect::>(), + vec![(bob.id(), 0)] + ); + } + + /// A backend that does not attest the scan state gets none of it, and a + /// load drops whatever it restored: the store is read from its start. + #[tokio::test] + async fn should_read_the_store_from_its_start_for_a_backend_without_the_capability() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let persistence = RecordingPersistence::attesting(PersistenceCapabilities::NONE); + let (_manager, bob_wallet) = wallet_on( + Arc::clone(&persistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![entry], + ) + .await; + let view = bob_wallet.identity().dashpay(); + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + view.ignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("ignored"); + view.unignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("un-ignored"); + assert!(persistence.contacts().iter().all(|changeset| { + changeset.private_contact_requests_positions.is_empty() + && changeset.private_shielded_addresses.is_empty() + && changeset.removed_private_shielded_addresses.is_empty() + })); + + let mut loaded = ManagedIdentity::new(bob.clone(), 0); + let key_fingerprint = loaded.private_scan_key_fingerprint(); + loaded.apply_private_contact_requests_position(PrivateContactRequestsPosition { + position: 5, + key_fingerprint, + private_payments_unresolved: false, + }); + loaded.apply_private_shielded_address(alice.id(), [0x55; 43]); + let mut start_state = IdentityManagerStartState::default(); + start_state + .wallet_identities + .entry(bob_wallet.wallet_id()) + .or_default() + .insert(0, loaded); + start_state.forget_private_contact_scan_state(); + let forgotten = &start_state.wallet_identities[&bob_wallet.wallet_id()][&0]; + assert_eq!(forgotten.dashpay().private_contact_requests_position(), 0); + assert_eq!( + forgotten.dashpay().private_shielded_address(&alice.id()), + None + ); + } + + /// A backend declaring the scan state without atomic changesets, which an + /// un-ignore's removal and rewind need to commit together, is treated as + /// keeping none of it: it gets none of the writes, a load drops the + /// position it restored, and a private payment, whose marker it could not + /// keep, is refused. + #[tokio::test] + async fn should_keep_no_scan_state_for_a_backend_without_atomic_changesets() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let persistence = RecordingPersistence::attesting( + PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE, + ); + let (_manager, bob_wallet) = wallet_on( + Arc::clone(&persistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![entry], + ) + .await; + let view = bob_wallet.identity().dashpay(); + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + view.ignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("ignored"); + view.unignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("un-ignored"); + assert!(persistence.contacts().iter().all(|changeset| { + changeset.private_contact_requests_positions.is_empty() + && changeset.private_shielded_addresses.is_empty() + })); + let refused = view + .send_private_contact_request( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await; + assert!( + matches!( + refused, + Err(PlatformWalletError::PrivateContactRecoveryUnsupported) + ), + "{refused:?}" + ); + + // The un-ignore rewound the cursor: read on again, still storing nothing. + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store again"); + assert!(persistence + .contacts() + .iter() + .all(|changeset| changeset.private_contact_requests_positions.is_empty())); + let mut kept = scan_state_snapshot(&bob_wallet, &bob.id(), false).await; + kept.capabilities = PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE; + assert_eq!( + kept.identities[&0] + .dashpay() + .private_contact_requests_position(), + 1, + "the backend hands a position back" + ); + let manager = PlatformWalletManager::new( + Arc::new(dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await), + Arc::new(kept), + Arc::new(NoopEventHandler), + ); + manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + let reloaded = manager + .get_wallet(&bob_wallet.wallet_id()) + .await + .expect("the relaunched wallet"); + managed(&reloaded, BOB, |managed| { + assert_eq!( + managed.dashpay().private_contact_requests_position(), + 0, + "dropped" + ); + }) + .await; + } + + /// A hook a [`HookedCrypto`] runs before one of its calls. + type Hook = Arc std::pin::Pin + Send>> + Send + Sync>; + + /// A provider deriving from a test seed that runs hooks first: before + /// deriving a receiving xpub, as a slow or interactive signer pauses a + /// send's preparation, before scanning pages, and before scanning a + /// single page, as a restoration does. + struct HookedCrypto { + inner: SeedCryptoProvider, + before_receiving_xpub: Option, + before_scan: Option, + before_page_scan: Option, + } + + impl HookedCrypto { + fn new(phrase: &str) -> Self { + Self { + inner: SeedCryptoProvider::from_seed(seed(phrase), Network::Testnet), + before_receiving_xpub: None, + before_scan: None, + before_page_scan: None, + } + } + } + + #[async_trait::async_trait] + impl ContactCryptoProvider for HookedCrypto { + async fn receiving_xpub( + &self, + path: &DerivationPath, + ) -> Result { + if let Some(hook) = &self.before_receiving_xpub { + hook().await; + } + self.inner.receiving_xpub(path).await + } + + async fn ecdh_shared_secret( + &self, + path: &DerivationPath, + peer: &PublicKey, + ) -> Result, PlatformWalletError> { + self.inner.ecdh_shared_secret(path, peer).await + } + + async fn export_auto_accept_private_key( + &self, + path: &DerivationPath, + ) -> Result { + self.inner.export_auto_accept_private_key(path).await + } + + async fn account_reference( + &self, + path: &DerivationPath, + compact_xpub: &[u8], + account_index: u32, + version: u32, + ) -> Result { + self.inner + .account_reference(path, compact_xpub, account_index, version) + .await + } + + async fn unmask_account_reference( + &self, + path: &DerivationPath, + compact_xpub: &[u8], + account_reference: u32, + ) -> Result<(u32, u32), PlatformWalletError> { + self.inner + .unmask_account_reference(path, compact_xpub, account_reference) + .await + } + + async fn contact_info_seal( + &self, + root_path: &DerivationPath, + derivation_index: u32, + contact_id: &[u8; 32], + private_data_plaintext: &[u8], + private_data_iv: &[u8; 16], + ) -> Result + { + self.inner + .contact_info_seal( + root_path, + derivation_index, + contact_id, + private_data_plaintext, + private_data_iv, + ) + .await + } + + async fn contact_info_open( + &self, + root_path: &DerivationPath, + derivation_index: u32, + enc_to_user_id: &[u8; 32], + private_data_blob: &[u8], + ) -> Result + { + self.inner + .contact_info_open( + root_path, + derivation_index, + enc_to_user_id, + private_data_blob, + ) + .await + } + + async fn scan_private_document_page( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + start_position: u64, + page: &PrivateDocumentEntries, + ) -> Result { + if let Some(hook) = &self.before_page_scan { + hook().await; + } + self.inner + .scan_private_document_page( + keys, + data_contract_id, + document_type_name, + start_position, + page, + ) + .await + } + + async fn scan_private_document_pages( + &self, + keys: &PrivateDocumentKeyPaths, + data_contract_id: Identifier, + document_type_name: &str, + pages: &[(u64, &PrivateDocumentEntries)], + ) -> Result, PlatformWalletError> { + if let Some(hook) = &self.before_scan { + hook().await; + } + self.inner + .scan_private_document_pages(keys, data_contract_id, document_type_name, pages) + .await + } + + #[allow(clippy::too_many_arguments)] + async fn seal_private_contact_request( + &self, + sender_encryption_key_path: &DerivationPath, + dashpay_contract: &DataContract, + sender: &Identity, + sender_encryption_key_id: KeyID, + signing_key_id: KeyID, + signature: &[u8; 65], + recipient: &Identity, + recipient_key_id: KeyID, + request: &PrivateContactRequest, + esk_seed: &[u8; 32], + platform_version: &PlatformVersion, + ) -> Result { + self.inner + .seal_private_contact_request( + sender_encryption_key_path, + dashpay_contract, + sender, + sender_encryption_key_id, + signing_key_id, + signature, + recipient, + recipient_key_id, + request, + esk_seed, + platform_version, + ) + .await + } + } + + /// A hook that, the `call`th time it runs (from 1), says so on `reached` + /// and waits for `release`. + fn pause_on_call( + call: usize, + reached: Arc, + release: Arc, + ) -> Hook { + let runs = Arc::new(AtomicUsize::new(0)); + Arc::new(move || { + let (reached, release, runs) = ( + Arc::clone(&reached), + Arc::clone(&release), + Arc::clone(&runs), + ); + Box::pin(async move { + if runs.fetch_add(1, Ordering::SeqCst) + 1 == call { + reached.notify_one(); + release.notified().await; + } + }) + }) + } + + /// A public request that passed its channel check and is still being + /// prepared holds the pair: a private request started meanwhile is + /// refused before anything is paid, and only the public one goes on. + /// Once it ends, the pair is free again. + #[tokio::test] + async fn should_let_only_one_channel_send_to_a_pair_at_a_time() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + let reached = Arc::new(tokio::sync::Notify::new()); + let release = Arc::new(tokio::sync::Notify::new()); + // The second derivation: the first checks the provider's seed before + // the store is read. + let mut pausing = HookedCrypto::new(ALICE_PHRASE); + pausing.before_receiving_xpub = + Some(pause_on_call(2, Arc::clone(&reached), Arc::clone(&release))); + let paid = Arc::new(Mutex::new(Vec::new())); + + let (alice_id, bob_id) = (alice.id(), bob.id()); + let public = view.send_contact_request_with_external_signer( + &alice_id, + &bob_id, + None, + AutoAcceptProofSource::None, + &alice_signer, + &pausing, + ); + let private = async { + reached.notified().await; + let claimed = managed(&alice_wallet, ALICE, |managed| { + managed.dashpay().public_contact_request_claimed(&bob.id()) + }) + .await; + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&paid), + ) + .await; + release.notify_one(); + (claimed, sent) + }; + let (public, (claimed, private)) = tokio::join!(public, private); + assert!(claimed, "the paused public send holds the pair"); + assert!( + matches!(&private, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if *recipient == bob.id()), + "{private:?}" + ); + assert!(paid.lock().expect("paid").is_empty(), "nothing is paid"); + assert!( + !matches!( + &public, + Err(PlatformWalletError::PrivateContactRequestPending(_) + | PlatformWalletError::PrivateContactRequestNeedsPrivateReply(_)) + ), + "the public send goes on to its submission, which this mock refuses: {public:?}" + ); + + assert!(!pending_to(&alice_wallet, ALICE, &bob).await); + managed(&alice_wallet, ALICE, |managed| { + assert!(!managed.dashpay().public_contact_request_claimed(&bob.id())); + }) + .await; + let kept = Arc::new(Mutex::new(Vec::new())); + let after = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + after, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{after:?}" + ); + assert_eq!( + kept.lock().expect("kept").len(), + 1, + "the pair is free again" + ); + } + + /// `identity` without its key `key_id`, as a device that has not yet + /// fetched a key another device added holds it. + fn without_key(identity: &Identity, key_id: KeyID) -> Identity { + Identity::V0(IdentityV0 { + id: identity.id(), + public_keys: identity + .public_keys() + .iter() + .filter(|(id, _)| **id != key_id) + .map(|(id, key)| (*id, key.clone())) + .collect(), + balance: identity.balance(), + revision: identity.revision(), + }) + } + + /// A store read with keys missing one an entry is sealed to is read again + /// once a refresh brings the key: the refresh folds it in, the read + /// position and the read-to-the-end flag are rewound, and the next sync + /// records the request. + #[tokio::test] + async fn should_read_the_store_again_once_an_identity_gains_a_scanning_key() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let stale = without_key(&bob, DECRYPTION_KEY_ID); + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &stale, &[&alice, &bob], vec![entry]).await; + let view = bob_wallet.identity().dashpay(); + + let report = view + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store with his older keys"); + assert_eq!( + report.requests, + vec![], + "the entry opens with no key he holds" + ); + assert!(store_read(&bob_wallet, BOB).await); + managed(&bob_wallet, BOB, |managed| { + assert_eq!(managed.dashpay().private_contact_requests_position(), 1); + }) + .await; + + bob_wallet + .identity() + .refresh_identity(&bob.id()) + .await + .expect("the refresh folds in the decryption key"); + view.reconcile_private_scan_keys() + .await + .expect("reconciled"); + assert!( + !store_read(&bob_wallet, BOB).await, + "a send reads the store again" + ); + managed(&bob_wallet, BOB, |managed| { + assert_eq!(managed.dashpay().private_contact_requests_position(), 0); + }) + .await; + + let report = view + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store again"); + assert_eq!(report.requests.len(), 1, "{:?}", report.requests); + managed(&bob_wallet, BOB, |managed| { + assert!(managed.dashpay().incoming_request(&alice.id()).is_some()); + }) + .await; + } + + /// A pass whose provider resolves another wallet's seed, valid but + /// deriving unrelated keys that open nothing, is refused with nothing + /// read or recorded; the pass with the wallet's own seed then reads the + /// request the refused one would have passed over. + #[tokio::test] + async fn should_refuse_a_pass_with_another_seed_and_leave_its_position_unmoved() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry]).await; + let view = bob_wallet.identity().dashpay(); + + let another_seed = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let refused = view.sync_private_contact_requests(&another_seed, 4).await; + assert!( + matches!(refused, Err(PlatformWalletError::SeedMismatch { .. })), + "{refused:?}" + ); + managed(&bob_wallet, BOB, |managed| { + assert_eq!( + managed.dashpay().private_contact_requests_position(), + 0, + "nothing recorded as read" + ); + assert!(!managed.dashpay().private_contact_requests_read()); + assert!(managed.dashpay().incoming_request(&alice.id()).is_none()); + }) + .await; + + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("the pass with Bob's seed"); + managed(&bob_wallet, BOB, |managed| { + assert!(managed.dashpay().incoming_request(&alice.id()).is_some()); + assert_eq!(managed.dashpay().private_contact_requests_position(), 1); + }) + .await; + } + + /// A pass that started with older keys commits no progress once a key + /// update lands under it, even when the rewind that update brings is + /// taken while the pass runs: the next pass reads with the new keys from + /// where the update left the position. + #[tokio::test] + async fn should_not_advance_over_a_key_update_during_a_pass() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let stale = without_key(&bob, DECRYPTION_KEY_ID); + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &stale, &[&alice, &bob], vec![entry]).await; + + let mut updating = HookedCrypto::new(BOB_PHRASE); + updating.before_scan = Some({ + let bob_wallet = Arc::clone(&bob_wallet); + let bob = bob.clone(); + Arc::new(move || { + let (bob_wallet, bob) = (Arc::clone(&bob_wallet), bob.clone()); + Box::pin(async move { + let iw = bob_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&bob_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob"); + // A refresh's key, and a read before a send rewinding for it. + managed.identity = bob; + managed.reconcile_private_scan_keys(); + }) + }) + }); + bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&updating, 4) + .await + .expect("the pass with older keys"); + managed(&bob_wallet, BOB, |managed| { + assert_eq!( + managed.dashpay().private_contact_requests_position(), + 0, + "no progress over the update" + ); + assert!(!managed.dashpay().private_contact_requests_read()); + assert!(managed.dashpay().incoming_request(&alice.id()).is_none()); + }) + .await; + + bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("the pass with the new keys"); + managed(&bob_wallet, BOB, |managed| { + assert!(managed.dashpay().incoming_request(&alice.id()).is_some()); + assert_eq!(managed.dashpay().private_contact_requests_position(), 1); + }) + .await; + } + + /// A reply dropped while its payment source is still being resolved, as + /// a host's prompt left open and abandoned, keeps no reservation: + /// nothing was paid, and a retry goes on to pay. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_release_a_reply_dropped_while_its_payment_is_resolved() { + use crate::wallet::platform_wallet::PrivateReplyPayment; + + struct NoProver; + impl dpp::shielded::builder::OrchardProver for NoProver { + fn proving_key(&self) -> &grovedb_commitment_tree::ProvingKey { + panic!("nothing is proved") + } + } + + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (entry, received) = alice_to_bob([0x55; 43]).await; + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry]).await; + bob_wallet + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + + let dropped = tokio::time::timeout( + Duration::from_millis(200), + bob_wallet.accept_contact_request_replying_in_kind( + &received, + None, + &bob_signer, + &bob_crypto, + || { + std::future::pending::< + Result>, PlatformWalletError>, + >() + }, + ), + ) + .await; + assert!(dropped.is_err(), "the payment source never resolves"); + assert!(!pending_to(&bob_wallet, BOB, &alice).await); + + let reply = Arc::new(Mutex::new(Vec::new())); + let retried = bob_wallet + .identity() + .dashpay() + .accept_private_contact_request( + &received, + None, + &bob_signer, + &bob_crypto, + keep_entry(&reply), + ) + .await; + assert!( + matches!( + retried, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {retried:?}" + ); + assert_eq!(reply.lock().expect("reply").len(), 1, "the retry pays"); + } + + /// An endpoint that kept the submitted payment and answered with a + /// consensus verdict proves nothing, so the payment settles unconfirmed + /// with its journal record saved: a public request to the pair, which + /// would disclose it, and a replacement private payment, which would fund + /// a second request, are both refused while the original may execute. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_keep_a_pair_held_when_an_endpoint_refuses_its_payment() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let dir = tempfile::tempdir().expect("temporary shielded store"); + let store = Arc::new(tokio::sync::RwLock::new( + FileBackedShieldedStore::open_path(dir.path().join("tree.sqlite"), 100) + .expect("the store"), + )); + let subwallet = SubwalletId::new(alice_wallet.wallet_id(), 0); + let pay = { + let store = Arc::clone(&store); + let (sender, recipient) = (alice.id(), bob.id()); + move |entry: BinaryData, started: PaymentStarted| async move { + let transition = private_contact_request_payment(entry); + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x7A; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "private document create from shielded pool", + Some(PrivateContactPayment { + sender: sender.to_buffer(), + recipient: recipient.to_buffer(), + account_reference: started + .account_reference() + .expect("the send names its request"), + }), + Some(&started), + 0, + ) + .await?; + // What the payment returns once the endpoint answered with a + // serialized consensus verdict: unconfirmed. + Err::(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: "the endpoint answered with a consensus verdict".to_string(), + }) + } + }; + let view = alice_wallet.identity().dashpay(); + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{sent:?}" + ); + assert_eq!( + store + .read() + .await + .private_payment_journal(alice_wallet.wallet_id()) + .expect("the journal") + .len(), + 1, + "the journal record stays" + ); + let public = view + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "no disclosure: {public:?}" + ); + let replacement = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + no_payment, + ) + .await; + assert!( + matches!( + replacement, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "no replacement funding: {replacement:?}" + ); + } + + /// A restoration that read a payment's journal record before the record + /// was discharged and its reservation released, as a never-delivered + /// payment does, and applies only after, does not reserve the pair again: + /// no entry would ever settle that reservation. A retry to the pair pays. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_not_restore_a_payment_discharged_while_the_restore_waited() { + use std::future::Future; + use std::task::{Context, Waker}; + + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let dir = tempfile::tempdir().expect("temporary shielded store"); + let db_path = dir.path().join("tree.sqlite"); + let coordinator = Arc::new(NetworkShieldedCoordinator::new( + Arc::new(dash_sdk::Sdk::new_mock()), + Network::Testnet, + db_path.clone(), + FileBackedShieldedStore::open_path(&db_path, 100).expect("the store"), + )); + let subwallet = SubwalletId::new(alice_wallet.wallet_id(), 0); + let paid = Arc::new(Mutex::new(Vec::new())); + let pay = { + let store = Arc::clone(coordinator.store()); + let paid = Arc::clone(&paid); + let (sender, recipient) = (alice.id(), bob.id()); + move |entry: BinaryData, started: PaymentStarted| async move { + paid.lock().expect("paid").push(entry.clone()); + let transition = private_contact_request_payment(entry); + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x7B; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "private document create from shielded pool", + Some(PrivateContactPayment { + sender: sender.to_buffer(), + recipient: recipient.to_buffer(), + account_reference: started + .account_reference() + .expect("the send names its request"), + }), + Some(&started), + 0, + ) + .await?; + Err::(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: "the result wait timed out".to_string(), + }) + } + }; + let view = alice_wallet.identity().dashpay(); + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{sent:?}" + ); + let commitment: [u8; 32] = paid.lock().expect("paid")[0].as_slice()[..32] + .try_into() + .expect("an entry starts with its commitment"); + + let mut cx = Context::from_waker(Waker::noop()); + let held_store = coordinator.store().write().await; + let mut restore = Box::pin(alice_wallet.restore_private_contact_payments(&coordinator)); + assert!( + restore.as_mut().poll(&mut cx).is_pending(), + "waits for the store" + ); + let mut held_wallet = alice_wallet.wallet_manager().write().await; + drop(held_store); + // The restoration reads its journal snapshot, then waits for the wallet. + assert!( + restore.as_mut().poll(&mut cx).is_pending(), + "waits for the wallet" + ); + // Meanwhile the record is discharged and the reservation released. + coordinator + .store() + .write() + .await + .discharge_private_payment(alice_wallet.wallet_id(), &commitment) + .expect("discharged"); + alice_wallet + .persister() + .note_discharged_private_entries([commitment]); + held_wallet + .get_wallet_info_mut(&alice_wallet.wallet_id()) + .expect("the wallet") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("alice") + .release_discharged_private_replies(&BTreeSet::from([commitment])); + drop(held_wallet); + restore.await.expect("the restoration"); + assert!( + !pending_to(&alice_wallet, ALICE, &bob).await, + "not reserved again" + ); + + let retry = Arc::new(Mutex::new(Vec::new())); + let retried = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&retry), + ) + .await; + assert!( + matches!( + retried, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {retried:?}" + ); + assert_eq!(retry.lock().expect("retry").len(), 1, "the retry pays"); + } + + /// The legacy restoration from the redrive records + /// (`PlatformWallet::restore_pending_private_contact_requests`) that read + /// a payment's transition before its journal record was discharged, and + /// the discharge notice taken and its reservation released while it + /// waited to open the entry, does not reserve the pair again: it reserves + /// through the journal restoration's step, which checks the discharge + /// under the wallet lock. A retry to the pair pays. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_not_restore_from_a_redrive_a_payment_discharged_while_the_restore_waited() { + use std::future::Future; + use std::task::{Context, Waker}; + + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let fixture = ExpiryFixture::new().await; + let entry = fixture + .pay_unconfirmed(ExpiryFixture::ANCHOR, ExpiryFixture::ANCHOR_SET_HEIGHT) + .await; + let commitment: [u8; 32] = entry.as_slice()[..32] + .try_into() + .expect("an entry starts with its commitment"); + let wallet = &fixture.wallet; + + let mut cx = Context::from_waker(Waker::noop()); + let held_store = fixture.coordinator.store().write().await; + let mut restore = Box::pin( + wallet.restore_pending_private_contact_requests(&fixture.coordinator, &alice_crypto), + ); + assert!( + restore.as_mut().poll(&mut cx).is_pending(), + "waits for the store" + ); + let mut held_wallet = wallet.wallet_manager().write().await; + drop(held_store); + // The restoration reads the redrive records, then waits for the + // wallet before it opens the entry. + assert!( + restore.as_mut().poll(&mut cx).is_pending(), + "waits for the wallet" + ); + // Meanwhile the record is discharged, and the wallet takes the + // discharge notice and releases the reservation. + fixture + .coordinator + .store() + .write() + .await + .discharge_private_payment(wallet.wallet_id(), &commitment) + .expect("discharged"); + wallet + .persister() + .note_discharged_private_entries([commitment]); + let taken = wallet + .persister() + .take_unreleased_discharged_private_entries(); + assert_eq!(taken, BTreeSet::from([commitment]), "the notice is taken"); + held_wallet + .get_wallet_info_mut(&wallet.wallet_id()) + .expect("the wallet") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("alice") + .release_discharged_private_replies(&taken); + drop(held_wallet); + let restored = restore.await.expect("the restoration"); + assert!(restored.is_empty(), "{restored:?}"); + assert!(!pending_to(wallet, ALICE, &bob).await, "not reserved again"); + + let retry = Arc::new(Mutex::new(Vec::new())); + let retried = fixture.send_again(keep_entry(&retry)).await; + assert!( + matches!( + retried, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {retried:?}" + ); + assert_eq!(retry.lock().expect("retry").len(), 1, "the retry pays"); + } + + /// The check a private send makes right before it pays, over the journal + /// it read, does not hold the pair for a payment whose record was + /// discharged since: it reserves through the restorations' step. One + /// still journaled holds it. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_not_hold_a_pair_for_a_journaled_payment_discharged_since_the_read() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + for discharged in [true, false] { + let (_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let commitment = [0x5D; 32]; + if discharged { + alice_wallet + .persister() + .note_discharged_private_entries([commitment]); + } + let held = alice_wallet + .identity() + .dashpay() + .hold_for_journaled_private_payment(&alice.id(), &bob.id(), &[(0, commitment)]) + .await + .expect("the check"); + assert_eq!(held, !discharged, "discharged: {discharged}"); + assert_eq!( + pending_to(&alice_wallet, ALICE, &bob).await, + !discharged, + "discharged: {discharged}" + ); + } + } + + /// A restoration of a private payment's reservation through a handle + /// whose wallet was removed and re-created under its id reserves nothing + /// on the replacement: the step every restoration takes is fenced to the + /// handle's wallet generation. The replacement's own handle restores it. + #[tokio::test] + async fn should_not_restore_through_a_handle_whose_wallet_was_replaced() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (manager, old) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let wallet_id = old.wallet_id(); + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = old + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {sent:?}" + ); + let entry = kept.lock().expect("kept")[0].clone(); + + manager.remove_wallet(&wallet_id).await.expect("removed"); + let replacement = manager + .create_wallet_from_seed_bytes( + Network::Testnet, + &seed(ALICE_PHRASE), + WalletAccountCreationOptions::Default, + Some(0), + ) + .await + .expect("re-created"); + assert_eq!(replacement.wallet_id(), wallet_id); + { + let iw = replacement.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&wallet_id) + .expect("the replacement's info") + .identity_manager + .add_identity(alice.clone(), 0, wallet_id, &iw.persister) + .expect("alice"); + } + let stale = old + .identity() + .dashpay() + .restore_pending_private_contact_requests(std::slice::from_ref(&entry), &alice_crypto) + .await; + assert!( + matches!(stale, Err(PlatformWalletError::WalletNotFound(_))), + "{stale:?}" + ); + managed(&replacement, ALICE, |managed| { + assert!(!managed.dashpay().private_reply_pending(&bob.id())); + }) + .await; + let restored = replacement + .identity() + .dashpay() + .restore_pending_private_contact_requests(&[entry], &alice_crypto) + .await + .expect("restored"); + assert_eq!(restored, vec![(alice.id(), bob.id())]); + } + + /// Alice's wallet, bound to a coordinator over a new store, holding one + /// unspent input note: the private payment expiry tests pay from it. + #[cfg(feature = "shielded")] + struct ExpiryFixture { + _manager: PlatformWalletManager, + wallet: Arc, + coordinator: Arc, + subwallet: SubwalletId, + input: ShieldedNote, + db_path: std::path::PathBuf, + _dir: tempfile::TempDir, + } + + #[cfg(feature = "shielded")] + impl ExpiryFixture { + /// The proved height of the anchor set the expiry tests judge by. + const ANCHOR_SET_HEIGHT: u64 = 10; + /// The anchor the payment's record names. + const ANCHOR: [u8; 32] = [0x7C; 32]; + + async fn new() -> Self { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + // A backend that keeps the viewing key, so the wallet binds. + let (manager, wallet) = wallet_on( + RecordingPersistence::attesting( + PersistenceCapabilities::SHIELDED_FVK_RESTART + .union(PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART), + ), + ALICE_PHRASE, + &alice, + &[&alice, &bob], + vec![], + ) + .await; + let dir = tempfile::tempdir().expect("temporary shielded store"); + let db_path = dir.path().join("tree.sqlite"); + let coordinator = Arc::new(NetworkShieldedCoordinator::new( + Arc::clone(&wallet.identity().sdk), + Network::Testnet, + db_path.clone(), + FileBackedShieldedStore::open_path(&db_path, 100).expect("the store"), + )); + // Bound, the coordinator holds the wallet's persister, on which it + // notes the discharge. + wallet + .bind_shielded(&seed(ALICE_PHRASE), &[0], &coordinator) + .await + .expect("bound"); + let subwallet = SubwalletId::new(wallet.wallet_id(), 0); + let input = ShieldedNote { + position: 0, + cmx: [0x11; 32], + nullifier: [0x12; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + coordinator + .store() + .write() + .await + .save_note(subwallet, &input) + .expect("the input note"); + Self { + _manager: manager, + wallet, + coordinator, + subwallet, + input, + db_path, + _dir: dir, + } + } + + /// Every response the mock answers is proved at `height`. + fn prove_at(&self, height: u64) { + let mut sdk = (*self.wallet.identity().sdk).clone(); + sdk.mock().set_response_height(height); + } + + /// Send a private request from Alice to Bob whose payment, spending + /// the input note against `anchor` read from a proof at + /// `anchor_height`, is journaled and times out unconfirmed. Returns + /// the entry paid for. + async fn pay_unconfirmed(&self, anchor: [u8; 32], anchor_height: u64) -> BinaryData { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let paid = Arc::new(Mutex::new(Vec::new())); + let pay = { + let store = Arc::clone(self.coordinator.store()); + let paid = Arc::clone(&paid); + let subwallet = self.subwallet; + let nullifier = self.input.nullifier; + let (sender, recipient) = (alice.id(), bob.id()); + move |entry: BinaryData, started: PaymentStarted| async move { + paid.lock().expect("paid").push(entry.clone()); + let transition = private_contact_request_payment(entry); + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x7D; 32]), + anchor, + vec![nullifier], + &transition, + "private document create from shielded pool", + Some(PrivateContactPayment { + sender: sender.to_buffer(), + recipient: recipient.to_buffer(), + account_reference: started + .account_reference() + .expect("the send names its request"), + }), + Some(&started), + anchor_height, + ) + .await?; + Err::(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: "the result wait timed out".to_string(), + }) + } + }; + let sent = self + .wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ) + .await; + assert!( + matches!( + sent, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{sent:?}" + ); + let entry = paid.lock().expect("paid")[0].clone(); + entry + } + + /// The journal records a pass reads before it fetches the anchor set. + async fn journal_snapshot( + &self, + ) -> Vec<(WalletId, crate::wallet::shielded::PrivatePaymentRecord)> { + private_payment_journal_snapshot( + &*self.coordinator.store().read().await, + [self.wallet.wallet_id()], + ) + } + + /// The discharge a pass runs after a note scan covering + /// `scan_covered_height`, against `journal` and the anchor set + /// `recorded` proved at [`Self::ANCHOR_SET_HEIGHT`]. + async fn discharge( + &self, + journal: &[(WalletId, crate::wallet::shielded::PrivatePaymentRecord)], + recorded: &HashSet<[u8; 32]>, + scan_covered_height: Option, + ) { + self.coordinator + .discharge_expired_private_payments( + journal, + PrunedAnchorEvidence { + recorded, + height: Self::ANCHOR_SET_HEIGHT, + scan_covered_height, + }, + ) + .await; + } + + /// How many records the wallet's journal holds. + async fn journaled(&self) -> usize { + self.coordinator + .store() + .read() + .await + .private_payment_journal(self.wallet.wallet_id()) + .expect("the journal") + .len() + } + + /// Send Alice's request to Bob again with `pay`. + async fn send_again(&self, pay: F) -> Result + where + F: FnOnce(BinaryData, PaymentStarted) -> Fut + Send, + Fut: Future> + Send, + { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + self.wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ) + .await + } + + /// The record stays and the pair it holds stays held. + async fn assert_kept(&self, case: &str) { + assert_eq!(self.journaled().await, 1, "{case}: the record stays"); + let held = self.send_again(no_payment).await; + assert!( + matches!( + held, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{case}: {held:?}" + ); + } + + /// The record is discharged, durably, and a retry to the pair pays. + async fn assert_discharged(&self, case: &str) { + assert_eq!(self.journaled().await, 0, "{case}: discharged"); + let retry = Arc::new(Mutex::new(Vec::new())); + let retried = self.send_again(keep_entry(&retry)).await; + assert!( + matches!( + retried, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{case}: paid, not read back: {retried:?}" + ); + assert_eq!( + retry.lock().expect("retry").len(), + 1, + "{case}: the retry pays" + ); + let reopened = + FileBackedShieldedStore::open_path(&self.db_path, 100).expect("reopened"); + assert!( + reopened + .private_payment_journal(self.wallet.wallet_id()) + .expect("the journal") + .is_empty(), + "{case}: the reopened store keeps the discharge" + ); + } + } + + /// A private payment that never executes is proven expired once its + /// anchor is pruned and its entry is absent from the store: the + /// coordinator discharges its journal record, the pair is freed, a retry + /// to it pays, and a reopened store restores no hold. So it is when a + /// competing spend consumed its input first: the absent entry shows it + /// never executed. A still-recorded anchor or an entry in the store each + /// keep the record and the hold. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_free_a_pair_once_its_private_payment_provably_expired() { + #[derive(Clone, Copy, Debug, PartialEq)] + enum Case { + Expired, + AnchorRecorded, + InputSpent, + EntryAppended, + } + let height = ExpiryFixture::ANCHOR_SET_HEIGHT; + for case in [ + Case::Expired, + Case::AnchorRecorded, + Case::InputSpent, + Case::EntryAppended, + ] { + let fixture = ExpiryFixture::new().await; + fixture.prove_at(height); + let entry = fixture.pay_unconfirmed(ExpiryFixture::ANCHOR, height).await; + match case { + Case::InputSpent => { + fixture + .coordinator + .store() + .write() + .await + .mark_spent(fixture.subwallet, &fixture.input.nullifier) + .expect("spent"); + } + Case::EntryAppended => publish(&fixture.wallet, 0, vec![entry.to_vec()]).await, + Case::Expired | Case::AnchorRecorded => {} + } + let recorded: HashSet<[u8; 32]> = match case { + Case::AnchorRecorded => HashSet::from([ExpiryFixture::ANCHOR]), + _ => HashSet::new(), + }; + let journal = fixture.journal_snapshot().await; + fixture.discharge(&journal, &recorded, Some(height)).await; + if matches!(case, Case::Expired | Case::InputSpent) { + fixture.assert_discharged(&format!("{case:?}")).await; + } else { + fixture.assert_kept(&format!("{case:?}")).await; + } + } + } + + /// The expiry evidence is pinned to the anchor set's proved height H and + /// to the journal records read before it: a record journaled after the + /// snapshot, one armed again since, one whose anchor was read above H, a + /// note scan or a store read below H each discharge nothing. The payment + /// whose scan or store read was below H is discharged once both cover H, + /// and a retry to its pair pays. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_discharge_an_expired_private_payment_only_on_evidence_at_its_anchor_set_height() + { + #[derive(Clone, Copy, Debug, PartialEq)] + enum Case { + JournaledAfterSnapshot, + ArmedAgainSinceSnapshot, + AnchorReadAboveSet, + ScanBelowSet, + StoreBelowSet, + } + let height = ExpiryFixture::ANCHOR_SET_HEIGHT; + let recorded = HashSet::new(); + for case in [ + Case::JournaledAfterSnapshot, + Case::ArmedAgainSinceSnapshot, + Case::AnchorReadAboveSet, + Case::ScanBelowSet, + Case::StoreBelowSet, + ] { + let fixture = ExpiryFixture::new().await; + // Read before the anchor set: the payment is journaled after it. + let before = fixture.journal_snapshot().await; + fixture.prove_at(match case { + Case::StoreBelowSet => height - 1, + _ => height, + }); + let anchor_height = match case { + Case::AnchorReadAboveSet => height + 1, + _ => height, + }; + let entry = fixture + .pay_unconfirmed(ExpiryFixture::ANCHOR, anchor_height) + .await; + let journal = match case { + Case::JournaledAfterSnapshot => before, + _ => fixture.journal_snapshot().await, + }; + assert!( + case == Case::JournaledAfterSnapshot || journal.len() == 1, + "{case:?}: the snapshot holds the record" + ); + if case == Case::ArmedAgainSinceSnapshot { + // A retry armed the same entry again, against a newer anchor + // the set may not hold yet. + let payment = journal[0].1.payment; + arm_redrive_before_broadcast( + fixture.coordinator.store(), + fixture.subwallet, + Some([0x7D; 32]), + [0x7E; 32], + vec![fixture.input.nullifier], + &private_contact_request_payment(entry.clone()), + "private document create from shielded pool", + Some(payment), + None, + height, + ) + .await + .expect("armed again"); + } + let scan_covered_height = match case { + Case::ScanBelowSet => Some(height - 1), + _ => Some(height), + }; + fixture + .discharge(&journal, &recorded, scan_covered_height) + .await; + fixture.assert_kept(&format!("{case:?}")).await; + if matches!(case, Case::ScanBelowSet | Case::StoreBelowSet) { + // The same payment, once the scan and the store read cover H. + fixture.prove_at(height); + fixture.discharge(&journal, &recorded, Some(height)).await; + fixture + .assert_discharged(&format!("{case:?} at the set's height")) + .await; + } + } + } + + /// The store read for expiry keeps only the candidates it finds, however + /// many other entries it reads to show them absent. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_keep_only_candidate_commitments_from_the_store_read_for_expiry() { + let fixture = ExpiryFixture::new().await; + let held = [0xC1; 32]; + let absent = [0xC2; 32]; + let mut entries: Vec> = (0u32..2_000) + .map(|index| { + let mut entry = vec![0u8; 64]; + entry[..4].copy_from_slice(&index.to_be_bytes()); + entry[4] = 0xEE; + entry + }) + .collect(); + let mut candidate_entry = held.to_vec(); + candidate_entry.extend_from_slice(&[0u8; 32]); + entries.insert(1_234, candidate_entry); + publish(&fixture.wallet, 0, entries).await; + let kept = fixture + .coordinator + .private_contact_requests_held(&HashSet::from([held, absent]), 0) + .await + .expect("the store reads"); + assert_eq!(kept, HashSet::from([held])); + } + + /// A payment that may have reached the network keeps its pair pending + /// whatever error it returns, as its transition may still execute; one + /// whose transition provably never left the wallet releases it. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_keep_a_pair_pending_once_its_payment_may_have_reached_the_network() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + for never_left in [false, true] { + let (_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let failing = move |_: BinaryData, started: PaymentStarted| { + started.mark(); + if never_left { + started.note_never_delivered(); + } + std::future::ready(Err::( + PlatformWalletError::ShieldedBroadcastFailed( + "rejected on its merits".to_string(), + ), + )) + }; + let sent = alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + failing, + ) + .await; + assert!( + matches!(sent, Err(PlatformWalletError::ShieldedBroadcastFailed(_))), + "{sent:?}" + ); + assert_eq!( + pending_to(&alice_wallet, ALICE, &bob).await, + !never_left, + "never left the wallet: {never_left}" + ); + } + } + + /// The type 29 transition paying for `entry`, with placeholder actions: + /// as the shielded store keeps a payment's signed transition. + #[cfg(feature = "shielded")] + fn private_contact_request_payment( + entry: BinaryData, + ) -> dpp::state_transition::StateTransition { + use dpp::shielded::SerializedAction; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::methods::PrivateDocumentCreateFromShieldedPoolTransitionMethodsV0; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::PrivateDocumentCreateFromShieldedPoolTransition; + + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + dashpay_contract().expect("bundled DashPay").id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME.to_string(), + vec![entry], + vec![action.clone(), action], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition") + } + + /// A process that exits after its private payment was admitted, during + /// the wait for its result, still leaves the payment's record, saved + /// before the broadcast: a restarted wallet restores the request's + /// reservation from it, so neither a second private request nor a + /// public one goes to the recipient. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_restore_a_request_whose_process_exited_during_its_result_wait() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let dir = tempfile::tempdir().expect("temporary shielded store"); + let db_path = dir.path().join("tree.sqlite"); + let coordinator_at = |db_path: std::path::PathBuf| { + let store = FileBackedShieldedStore::open_path(&db_path, 100).expect("open the store"); + Arc::new(NetworkShieldedCoordinator::new( + Arc::new(dash_sdk::Sdk::new_mock()), + Network::Testnet, + db_path, + store, + )) + }; + + let before = coordinator_at(db_path.clone()); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let subwallet = SubwalletId::new(alice_wallet.wallet_id(), 0); + let pay = { + let store = Arc::clone(before.store()); + move |entry: BinaryData, started: PaymentStarted| async move { + // As the shielded payment does: the record saved, marked, + // broadcast and admitted; then the result never comes. + let transition = private_contact_request_payment(entry); + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x77; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "private document create from shielded pool", + None, + Some(&started), + 0, + ) + .await?; + std::future::pending::<()>().await; + Ok(ResponseMetadata::default()) + } + }; + let exited = tokio::time::timeout( + Duration::from_millis(200), + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + pay, + ), + ) + .await; + assert!(exited.is_err(), "the process exits during the wait"); + drop(before); + + let after = coordinator_at(db_path); + let (_restarted_manager, restarted) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + assert_eq!(restarted.wallet_id(), alice_wallet.wallet_id()); + assert!(!pending_to(&restarted, ALICE, &bob).await); + let restored = restarted + .restore_pending_private_contact_requests(&after, &alice_crypto) + .await + .expect("restored from the saved record"); + assert_eq!(restored, vec![(alice.id(), bob.id())]); + assert!(pending_to(&restarted, ALICE, &bob).await); + let public = restarted + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + &alice_signer, + &alice_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "{public:?}" + ); + } + + /// Two accepts of one private request straddling the first reply's + /// settlement: the second read that no reply was sent before the first + /// ran, and reaches its reply after the first's was recorded and its + /// reservation settled. It adopts that reply, and its payment never runs. + #[tokio::test] + async fn should_adopt_a_reply_another_accept_recorded_in_between() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (entry, received) = alice_to_bob([0x55; 43]).await; + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry]).await; + let view = bob_wallet.identity().dashpay(); + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + let first_done = Arc::new(tokio::sync::Notify::new()); + let reply = Arc::new(Mutex::new(Vec::new())); + + let second = view.accept_contact_request_replying_with(&received, &bob_crypto, || { + let first_done = Arc::clone(&first_done); + let (received, bob_signer, bob_crypto) = (&received, &bob_signer, &bob_crypto); + async move { + first_done.notified().await; + view.send_private_reply(received, None, bob_signer, bob_crypto, no_payment) + .await + } + }); + let first = async { + let accepted = view + .accept_private_contact_request( + &received, + None, + &bob_signer, + &bob_crypto, + append_entry(&bob_wallet, 1, vec![], &reply), + ) + .await; + first_done.notify_one(); + accepted + }; + let (second, first) = tokio::join!(second, first); + let first = first.expect("the first reply is paid and recorded"); + let second = second.expect("the second accept adopts it"); + assert_eq!(second.outgoing_request, first.outgoing_request); + assert_eq!(reply.lock().expect("reply").len(), 1, "one reply is paid"); + assert!(!pending_to(&bob_wallet, BOB, &alice).await); + } + + /// A `pay` that must not run. + fn no_payment( + _: BinaryData, + _: PaymentStarted, + ) -> std::future::Ready> { + panic!("nothing is paid") + } + + /// Whether `wallet`'s identity `identity_id` read the private store to its + /// end since load. + async fn store_read(wallet: &PlatformWallet, identity_id: [u8; 32]) -> bool { + managed(wallet, identity_id, |managed| { + managed.dashpay().private_contact_requests_read() + }) + .await + } + + /// A private reply paid for and confirmed but not read back leaves + /// nothing that survives a restart. After one, with Alice's request + /// already known (as a sync that stopped between the two leaves it), the + /// accept a host calls reads the store first, finds the reply, and adopts + /// it: nothing is paid twice. + #[tokio::test] + async fn should_adopt_a_private_reply_paid_for_before_a_restart() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let bob_signer = signer(BOB_PHRASE, &bob); + let (_alice_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the request"); + let request_entry = kept.lock().expect("kept").remove(0); + + let (_bob_manager, before) = wallet( + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![request_entry.to_vec()], + ) + .await; + before + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("Bob reads the store"); + let received = managed(&before, BOB, |managed| { + managed.dashpay().incoming_request(&alice.id()).cloned() + }) + .await + .expect("Bob holds Alice's request"); + let reply = Arc::new(Mutex::new(Vec::new())); + let paid = before + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &received, + None, + &bob_signer, + &bob_crypto, + keep_entry(&reply), + ) + .await; + assert!( + matches!( + paid, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid and confirmed, not read back: {paid:?}" + ); + let reply_entry = reply.lock().expect("reply").remove(0); + + // The restart: the reply is in the store, the reservation is gone. + let (_restarted_manager, restarted) = wallet( + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![request_entry.to_vec(), reply_entry.to_vec()], + ) + .await; + { + let iw = restarted.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&restarted.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob") + .add_incoming_contact_request(received.clone(), &iw.persister) + .expect("Alice's request"); + } + assert!(!store_read(&restarted, BOB).await); + let established = restarted + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &received, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await + .expect("the reply read back is adopted"); + assert_eq!(established.contact_identity_id, alice.id()); + assert_eq!(established.incoming_request, received); + assert!(established.outgoing_request.came_privately()); + assert!(store_read(&restarted, BOB).await); + assert!(!pending_to(&restarted, BOB, &alice).await); + } + + /// Track `request` as an incoming request of `wallet`'s identity + /// `identity_id`, as a restarted wallet restores the public ones. + async fn add_incoming( + wallet: &PlatformWallet, + identity_id: [u8; 32], + request: &ContactRequest, + ) { + let iw = wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&Identifier::from(identity_id)) + .expect("managed identity") + .add_incoming_contact_request(request.clone(), &iw.persister) + .expect("incoming request"); + } + + /// A private reply to a public request, paid for and not read back, + /// leaves nothing that survives a restart. After one, before any sync, + /// a public send to the contact and the accept a host calls each read + /// the store first and find the reply: the send is refused, as the + /// pair's requests went privately, and the accept adopts the reply. + /// Nothing is broadcast or paid again. + #[tokio::test] + async fn should_not_reply_publicly_after_a_restart_with_a_private_reply_unread() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let incoming = public_request(&alice, &bob, 3); + let (_before_manager, before) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + add_incoming(&before, BOB, &incoming).await; + let reply = Arc::new(Mutex::new(Vec::new())); + let paid = before + .identity() + .dashpay() + .accept_private_contact_request( + &incoming, + None, + &bob_signer, + &bob_crypto, + keep_entry(&reply), + ) + .await; + assert!( + matches!( + paid, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid and confirmed, not read back: {paid:?}" + ); + let reply_entry = reply.lock().expect("reply").remove(0).to_vec(); + + // Restarted with the reply in the store: a public send first. + let (_sending_manager, sending) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![reply_entry.clone()]).await; + add_incoming(&sending, BOB, &incoming).await; + let public = sending + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(contact)) + if contact == alice.id() + ), + "{public:?}" + ); + + // Restarted again: the accept a host calls first. + let (_accepting_manager, accepting) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![reply_entry]).await; + add_incoming(&accepting, BOB, &incoming).await; + let established = accepting + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await + .expect("the private reply read back is adopted"); + assert_eq!(established.contact_identity_id, alice.id()); + assert_eq!(established.incoming_request, incoming); + assert!(established.outgoing_request.came_privately()); + } + + /// After a restart, before any sync, the accept a host calls reads the + /// store before it chooses the reply: a public request since followed by + /// a private one from the same sender, unread, takes a private reply, + /// where choosing from the public request alone would pick a public one + /// and refuse it once the store is read. + #[tokio::test] + async fn should_reply_privately_in_kind_once_a_restart_reads_a_private_request() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let bob_signer = signer(BOB_PHRASE, &bob); + let public = public_request(&alice, &bob, 3); + + // Alice's public request, then a private one to Bob. + let (_alice_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + { + let iw = alice_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&alice_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("Alice") + .add_sent_contact_request(public.clone(), &iw.persister) + .expect("the public request"); + } + let kept = Arc::new(Mutex::new(Vec::new())); + alice_wallet + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the private request"); + let private_entry = kept.lock().expect("kept").remove(0).to_vec(); + + // Bob, restarted, holds the public request; the private one is in the + // store, unread. + let (_bob_manager, bob_wallet) = + wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![private_entry]).await; + add_incoming(&bob_wallet, BOB, &public).await; + let reply = Arc::new(Mutex::new(Vec::new())); + let accepted = bob_wallet + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &public, + None, + &bob_signer, + &bob_crypto, + keep_entry(&reply), + ) + .await; + assert!( + matches!( + accepted, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "a private reply, paid and not read back: {accepted:?}" + ); + assert_eq!(reply.lock().expect("reply").len(), 1, "one reply is paid"); + } + + /// A private reply to a public request whose payment is unconfirmed when + /// the process ends is saved with the request it pays for: binding the + /// restarted wallet to its shielded coordinator reserves the pair again, + /// with no host call and no keys. Before any sync, neither the accept a + /// host calls nor a public send then goes to the contact. Before the + /// bind, the relaunched wallet, which bound its shielded account before, + /// refuses both too, with nothing sent: the reservation is not restored + /// yet. Once the reply's entry is appended and read back, the hold + /// resolves. A payment whose broadcast may have been delivered is saved + /// and kept exactly so (`settle_private_document_payment`). + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_not_reply_publicly_after_a_restart_with_a_private_reply_unconfirmed() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let incoming = public_request(&alice, &bob, 3); + let dir = tempfile::tempdir().expect("temporary shielded store"); + let db_path = dir.path().join("tree.sqlite"); + let coordinator_at = |db_path: std::path::PathBuf| { + let store = FileBackedShieldedStore::open_path(&db_path, 100).expect("open the store"); + Arc::new(NetworkShieldedCoordinator::new( + Arc::new(dash_sdk::Sdk::new_mock()), + Network::Testnet, + db_path, + store, + )) + }; + + let before_coordinator = coordinator_at(db_path.clone()); + let (_before_manager, before) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + add_incoming(&before, BOB, &incoming).await; + // What the backend keeps of the wallet, before the reply is paid. + let kept = Arc::new(snapshot(&before, &bob.id(), BOB_PHRASE).await); + let subwallet = SubwalletId::new(before.wallet_id(), 0); + let paid_entries = Arc::new(Mutex::new(Vec::new())); + let pay = { + let store = Arc::clone(before_coordinator.store()); + let paid_entries = Arc::clone(&paid_entries); + let (sender, recipient) = (bob.id(), alice.id()); + move |entry: BinaryData, started: PaymentStarted| async move { + paid_entries.lock().expect("paid").push(entry.clone()); + // As the shielded payment does: the record saved with the + // request it pays for, marked, broadcast; its result unknown. + let transition = private_contact_request_payment(entry); + let payment = PrivateContactPayment { + sender: sender.to_buffer(), + recipient: recipient.to_buffer(), + account_reference: started + .account_reference() + .expect("the send names its request"), + }; + arm_redrive_before_broadcast( + &store, + subwallet, + Some([0x77; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "private document create from shielded pool", + Some(payment), + Some(&started), + 0, + ) + .await?; + Err::(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create", + reason: "the result wait timed out".to_string(), + }) + } + }; + let paid = before + .identity() + .dashpay() + .accept_private_contact_request(&incoming, None, &bob_signer, &bob_crypto, pay) + .await; + assert!( + matches!( + paid, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{paid:?}" + ); + drop(before_coordinator); + + // The restart: the reply is not in the store and the host restores + // nothing; only the bind can. + let after = coordinator_at(db_path); + let restarted_manager = PlatformWalletManager::new( + Arc::new(dashpay_sdk(&[&alice, &bob], vec![], PlatformVersion::latest(), None).await), + kept, + Arc::new(NoopEventHandler), + ); + restarted_manager + .load_from_persistor() + .await + .expect("the relaunch loads"); + let restarted = restarted_manager + .get_wallet(&before.wallet_id()) + .await + .expect("the relaunched wallet"); + assert!(!pending_to(&restarted, BOB, &alice).await); + + // Before the bind: refused, with nothing read or sent. + let accepted = restarted + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await; + assert!( + matches!( + accepted, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{accepted:?}" + ); + let public = restarted + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactPaymentsNotRestored) + ), + "{public:?}" + ); + + restarted + .bind_shielded(&seed(BOB_PHRASE), &[0], &after) + .await + .expect("bound"); + assert!( + pending_to(&restarted, BOB, &alice).await, + "the bind restores the reservation" + ); + // An identical rebind while a scan holds the store returns without + // waiting for it: the wallet is held until the restoration, deferred + // to a task, runs once the store frees, and the pair stays held. + { + let scan = after.store().write().await; + tokio::time::timeout( + Duration::from_secs(5), + restarted.bind_shielded(&seed(BOB_PHRASE), &[0], &after), + ) + .await + .expect("the rebind does not wait for the store") + .expect("rebound"); + assert!( + restarted.private_payments_held().await, + "held until the deferred restoration" + ); + drop(scan); + } + for _ in 0..100 { + if !restarted.private_payments_held().await { + break; + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert!( + !restarted.private_payments_held().await, + "the deferred restoration released the wallet" + ); + assert!(pending_to(&restarted, BOB, &alice).await); + + let accepted = restarted + .identity() + .dashpay() + .accept_contact_request_replying_in_kind( + &incoming, + None, + &bob_signer, + &bob_crypto, + no_payment, + ) + .await; + assert!( + matches!( + accepted, + Err(PlatformWalletError::PrivateContactRequestPending(contact)) + if contact == alice.id() + ), + "{accepted:?}" + ); + let public = restarted + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestPending(contact)) + if contact == alice.id() + ), + "{public:?}" + ); + + // Once the reply's entry is appended and read back, its reservation + // settles: the hold resolves. + let entry = paid_entries.lock().expect("paid").remove(0); + publish(&restarted, 0, vec![entry.to_vec()]).await; + restarted + .identity() + .dashpay() + .sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("the store is read"); + assert!( + !pending_to(&restarted, BOB, &alice).await, + "settled once read back" + ); + // Read back and recorded durably, its journal record is discharged: + // the next restoration finds none, and the pair stays free. + restarted + .restore_private_contact_payments(&after) + .await + .expect("restored"); + assert!( + after + .store() + .read() + .await + .private_payment_journal(restarted.wallet_id()) + .expect("the journal") + .is_empty(), + "discharged once read back" + ); + assert!(!pending_to(&restarted, BOB, &alice).await); + } + + /// A payment saved with its request is restored only while neither that + /// request nor a later one is recorded: a request read back before the + /// restart discharges it, a rotation past the tracked request does not, + /// and a sender that is not the wallet's identity is skipped. + #[cfg(feature = "shielded")] + #[tokio::test] + async fn should_restore_a_saved_payment_only_while_its_request_is_unrecorded() { + use platform_encryption::ACCOUNT_REFERENCE_VERSION_SHIFT; + + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + let reference = 5; + let rotated = reference | (1 << ACCOUNT_REFERENCE_VERSION_SHIFT); + { + let iw = bob_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&bob_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&bob.id()) + .expect("Bob") + .add_sent_contact_request(public_request(&bob, &alice, reference), &iw.persister) + .expect("sent request"); + } + let view = bob_wallet.identity().dashpay(); + + let recorded = view + .restore_private_payments( + &[ + (bob.id(), alice.id(), reference, [9; 32]), + (alice.id(), bob.id(), reference, [8; 32]), + ], + true, + ) + .await + .expect("restore"); + assert!(recorded.is_empty(), "{recorded:?}"); + assert!(!pending_to(&bob_wallet, BOB, &alice).await); + + let rotation = view + .restore_private_payments(&[(bob.id(), alice.id(), rotated, [9; 32])], true) + .await + .expect("restore"); + assert_eq!(rotation, vec![(bob.id(), alice.id())]); + assert!(pending_to(&bob_wallet, BOB, &alice).await); + } + + /// On a network a verified answer shows below protocol version 15 there + /// is no private store to read: a public send goes on to its broadcast, + /// as it did before private contact requests, rather than being refused + /// for a read that cannot run. + #[tokio::test] + async fn should_send_publicly_on_a_network_below_protocol_version_15() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (_manager, bob_wallet) = wallet_at( + Arc::new(NoPersistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![], + PlatformVersion::get(14).expect("protocol version 14"), + Some(14), + ) + .await; + let sent = bob_wallet + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + // Past the read and the claim: the transition is being built, and + // this mock does not answer its nonce fetch. + assert!( + matches!(&sent, Err(PlatformWalletError::Sdk(error)) + if error.to_string().contains("GetIdentityContractNonceRequest")), + "the send goes on to its transition: {sent:?}" + ); + assert!(!store_read(&bob_wallet, BOB).await, "nothing was read"); + } + + /// An SDK pinned below protocol version 15 refuses to read the private + /// store whatever the network runs. On a network a verified answer shows + /// at 15, or one whose version is not verified, the store may hold a + /// private request, so a public send is refused with that refusal and + /// sends nothing. + #[tokio::test] + async fn should_refuse_a_public_send_from_an_sdk_pinned_below_a_network_at_15() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + for verified_network in [Some(15), None] { + let (_manager, bob_wallet) = wallet_at( + Arc::new(NoPersistence), + BOB_PHRASE, + &bob, + &[&alice, &bob], + vec![], + PlatformVersion::get(14).expect("protocol version 14"), + verified_network, + ) + .await; + let sent = bob_wallet + .identity() + .dashpay() + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + &sent, + Err(PlatformWalletError::Sdk( + dash_sdk::Error::ProtocolVersionNotActive { .. } + )) + ), + "verified {verified_network:?}: {sent:?}" + ); + } + } + + /// A public request's claim is released only by its own attempt: a send + /// that outlived its wallet's removal and re-creation under the same id, + /// dropped once the new wallet's send claimed the same pair, leaves that + /// claim in place, and the new send's own release frees the pair. + #[tokio::test] + async fn should_release_only_its_own_claim_across_a_wallet_re_creation() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let (manager, old_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let old_claim = old_wallet + .identity() + .dashpay() + .claim_public_pair(&alice.id(), &bob.id(), false) + .await + .expect("claimed") + .expect("a claim"); + + manager + .remove_wallet(&old_wallet.wallet_id()) + .await + .expect("removed"); + let new_wallet = manager + .create_wallet_from_seed_bytes( + Network::Testnet, + &seed(ALICE_PHRASE), + WalletAccountCreationOptions::Default, + Some(0), + ) + .await + .expect("re-created"); + assert_eq!(new_wallet.wallet_id(), old_wallet.wallet_id()); + { + let iw = new_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&new_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .add_identity(alice.clone(), 0, new_wallet.wallet_id(), &iw.persister) + .expect("own identity"); + } + let new_claim = new_wallet + .identity() + .dashpay() + .claim_public_pair(&alice.id(), &bob.id(), false) + .await + .expect("claimed") + .expect("a claim"); + let claimed = || async { + managed(&new_wallet, ALICE, |managed| { + managed.dashpay().public_contact_request_claimed(&bob.id()) + }) + .await + }; + + drop(old_claim); + assert!( + claimed().await, + "the old send's release leaves the new claim" + ); + drop(new_claim); + assert!(!claimed().await, "the new send's release frees the pair"); + } + + /// Unignoring a sender rewinds the private read and clears its + /// readiness: a pass that read the store with the sender ignored passed + /// over its private request, so a public send to it reads the store + /// again first, finds the request, and is refused, publishing nothing. + #[tokio::test] + async fn should_read_the_store_again_before_a_public_send_once_a_sender_is_unignored() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (entry, _) = alice_to_bob([0x55; 43]).await; + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![entry]).await; + let view = bob_wallet.identity().dashpay(); + view.ignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("ignored"); + view.sync_private_contact_requests(&bob_crypto, 4) + .await + .expect("the pass"); + assert!(store_read(&bob_wallet, BOB).await); + assert!( + managed(&bob_wallet, BOB, |managed| { + managed.dashpay().incoming_request(&alice.id()).is_none() + }) + .await, + "passed over while ignored" + ); + + view.unignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("unignored"); + assert!(!store_read(&bob_wallet, BOB).await, "read again first"); + let public = view + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(contact)) + if contact == alice.id() + ), + "{public:?}" + ); + } + + /// A sender unignored while a pass runs, after the pass passed over its + /// private request: the pass reaches the store's end without taking the + /// store as read, so a public send to the sender reads it again first, + /// finds the request, and is refused, publishing nothing. + #[tokio::test] + async fn should_not_take_the_store_as_read_by_a_pass_a_sender_was_unignored_under() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let bob_crypto = SeedCryptoProvider::from_seed(seed(BOB_PHRASE), Network::Testnet); + let bob_signer = signer(BOB_PHRASE, &bob); + let (entry, _) = alice_to_bob([0x55; 43]).await; + // Alice's request first, then pages enough for a second scan batch. + let mut entries = vec![entry]; + entries.extend((1..=PRIVATE_SCAN_BATCH_PAGES as u8).map(|byte| vec![byte; 589])); + let (_manager, bob_wallet) = wallet(BOB_PHRASE, &bob, &[&alice, &bob], vec![]).await; + publish_chunked(&bob_wallet, &entries, 1).await; + let view = bob_wallet.identity().dashpay(); + view.ignore_contact_sender(&bob.id(), &alice.id()) + .await + .expect("ignored"); + + // Unignored before the pass scans its second batch, once the first, + // holding Alice's request, was ingested with her ignored. + let mut unignoring = HookedCrypto::new(BOB_PHRASE); + unignoring.before_scan = Some({ + let bob_wallet = Arc::clone(&bob_wallet); + let scans = Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let (bob_id, alice_id) = (bob.id(), alice.id()); + Arc::new(move || { + let (bob_wallet, scans) = (Arc::clone(&bob_wallet), Arc::clone(&scans)); + Box::pin(async move { + if scans.fetch_add(1, Ordering::SeqCst) == 1 { + bob_wallet + .identity() + .dashpay() + .unignore_contact_sender(&bob_id, &alice_id) + .await + .expect("unignored"); + } + }) + }) + }); + view.sync_private_contact_requests(&unignoring, 64) + .await + .expect("the pass"); + assert!( + !store_read(&bob_wallet, BOB).await, + "not read for a sender unignored under the pass" + ); + + let public = view + .send_contact_request_with_external_signer( + &bob.id(), + &alice.id(), + None, + AutoAcceptProofSource::None, + &bob_signer, + &bob_crypto, + ) + .await; + assert!( + matches!( + public, + Err(PlatformWalletError::PrivateContactRequestNeedsPrivateReply(contact)) + if contact == alice.id() + ), + "{public:?}" + ); + } + + /// The first private send since load reads the store first. While that + /// read cannot finish, here because the recipient of a request the store + /// holds from Alice cannot be read, it is refused with nothing sealed or + /// paid; a sync error comes back as it is; once the store reads, the send + /// goes on. + #[tokio::test] + async fn should_refuse_a_private_send_until_the_store_reads_to_its_end() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, before) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let kept = Arc::new(Mutex::new(Vec::new())); + before + .identity() + .dashpay() + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry(&kept), + ) + .await + .expect_err("not read back"); + let entry = kept.lock().expect("kept").remove(0); + + let (_restarted_manager, restarted) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![entry.to_vec()]).await; + let view = restarted.identity().dashpay(); + macro_rules! send { + ($pay:expr) => { + view.send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + $pay, + ) + .await + }; + } + + set_identity_known(&restarted, &bob, false).await; + let held = send!(no_payment); + assert!( + matches!(held, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == bob.id()), + "{held:?}" + ); + assert!(!store_read(&restarted, ALICE).await); + assert!( + !pending_to(&restarted, ALICE, &bob).await, + "nothing reserved" + ); + + let contract = dashpay_contract().expect("bundled DashPay"); + let page = PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + ); + { + let mut sdk = (*restarted.identity().sdk).clone(); + sdk.mock() + .remove_fetch_expectation::(page) + .await; + } + let failed = send!(no_payment); + assert!( + matches!(failed, Err(PlatformWalletError::Sdk(_))), + "{failed:?}" + ); + + publish(&restarted, 0, vec![entry.to_vec()]).await; + set_identity_known(&restarted, &bob, true).await; + let paid = Arc::new(Mutex::new(Vec::new())); + let sent = send!(keep_entry(&paid)); + assert!( + matches!( + sent, + Err(PlatformWalletError::PrivateContactRequestPending(_)) + ), + "paid, not read back: {sent:?}" + ); + assert_eq!(paid.lock().expect("paid").len(), 1, "the send goes on"); + assert!(store_read(&restarted, ALICE).await); + managed(&restarted, ALICE, |managed| { + assert!( + managed.dashpay().outgoing_request(&bob.id()).is_some(), + "the request paid for before the restart is recorded first" + ); + }) + .await; + } + + /// Once the store was read to its end, a private send reads nothing more + /// before it pays: here a page read would fail, and the send still pays. + #[tokio::test] + async fn should_read_the_private_store_once_since_load() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let carol = identity(BOB_PHRASE, [0xC1; 32]); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = + wallet(ALICE_PHRASE, &alice, &[&alice, &bob, &carol], vec![]).await; + let view = alice_wallet.identity().dashpay(); + macro_rules! send { + ($recipient:expr, $kept:expr) => { + view.send_private_contact_request( + &alice.id(), + &$recipient, + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + keep_entry($kept), + ) + .await + }; + } + assert!(!store_read(&alice_wallet, ALICE).await); + let read_when_paid = Arc::new(AtomicBool::new(false)); + view.send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + |_: BinaryData, started: PaymentStarted| { + started.mark(); + let read_when_paid = Arc::clone(&read_when_paid); + let wallet = &alice_wallet; + async move { + read_when_paid.store(store_read(wallet, ALICE).await, Ordering::SeqCst); + Ok(ResponseMetadata { + time_ms: BLOCK_TIME_MS, + ..Default::default() + }) + } + }, + ) + .await + .expect_err("not read back"); + assert!( + read_when_paid.load(Ordering::SeqCst), + "the store was read before the first payment" + ); + + let contract = dashpay_contract().expect("bundled DashPay"); + { + let mut sdk = (*alice_wallet.identity().sdk).clone(); + sdk.mock() + .remove_fetch_expectation::( + PrivateDocumentEntriesQuery::new( + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + ), + ) + .await; + } + let second = Arc::new(Mutex::new(Vec::new())); + let sent = send!(carol.id(), &second); + assert!( + matches!(sent, Err(PlatformWalletError::PrivateContactRequestPending(recipient)) + if recipient == carol.id()), + "paid, its read back failing: {sent:?}" + ); + assert_eq!(second.lock().expect("second").len(), 1, "no read first"); + } + + /// Once the account reference's rotation version is at its last, a + /// request with the same xpub and key would repeat the tracked one: it is + /// refused before anything is paid, and nothing stays pending. + #[tokio::test] + async fn should_refuse_a_request_once_every_rotation_version_is_used() { + let alice = identity(ALICE_PHRASE, ALICE); + let bob = identity(BOB_PHRASE, BOB); + let alice_crypto = SeedCryptoProvider::from_seed(seed(ALICE_PHRASE), Network::Testnet); + let alice_signer = signer(ALICE_PHRASE, &alice); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[&alice, &bob], vec![]).await; + let view = alice_wallet.identity().dashpay(); + let kept = Arc::new(Mutex::new(Vec::new())); + let sent = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + append_entry(&alice_wallet, 0, vec![], &kept), + ) + .await + .expect("Alice sends the request, read back"); + assert_eq!(account_reference_version(sent.account_reference), 0); + + let mut exhausted = sent.clone(); + exhausted.account_reference = (sent.account_reference & 0x0FFF_FFFF) | (15 << 28); + assert_eq!(account_reference_version(exhausted.account_reference), 15); + { + let iw = alice_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + wm.get_wallet_info_mut(&alice_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("Alice") + .add_sent_contact_request(exhausted, &iw.persister) + .expect("the version 15 request"); + } + + let result = view + .send_private_contact_request( + &alice.id(), + &bob.id(), + None, + AutoAcceptProofSource::None, + None, + &alice_signer, + &alice_crypto, + |_: BinaryData, _: PaymentStarted| -> std::future::Ready> { + panic!("a request that repeats the tracked one is not paid for") + }, + ) + .await; + assert!( + matches!(&result, Err(PlatformWalletError::InvalidParameter(message)) + if message.contains("no account reference version is left")), + "{result:?}" + ); + assert!(!pending_to(&alice_wallet, ALICE, &bob).await); + } + + /// A cursor rewound under a pass is not advanced again that pass, so the + /// next reads from the rewind. + #[tokio::test] + async fn should_keep_a_cursor_rewound_under_a_pass() { + let alice = identity(ALICE_PHRASE, ALICE); + let (_manager, alice_wallet) = wallet(ALICE_PHRASE, &alice, &[], vec![]).await; + let iw = alice_wallet.identity(); + let mut wm = iw.wallet_manager.write().await; + let managed = wm + .get_wallet_info_mut(&alice_wallet.wallet_id()) + .expect("wallet info") + .identity_manager + .managed_identity_mut(&alice.id()) + .expect("Alice"); + let mut scanned = ScannedIdentity { + key_fingerprint: managed.private_scan_key_fingerprint(), + identity_id: alice.id(), + identity_index: 0, + position: 0, + failed: false, + rewound: false, + }; + scanned.advance(managed, 100); + assert_eq!(managed.dashpay().private_contact_requests_position(), 100); + // Unignoring a sender rewinds the cursor while a page is read. + let carol = Identifier::from([0xC1; 32]); + managed.ignore_sender(&carol); + managed.unignore_sender(&carol); + assert_eq!(managed.dashpay().private_contact_requests_position(), 0); + scanned.advance(managed, 200); + scanned.advance(managed, 300); + assert_eq!( + managed.dashpay().private_contact_requests_position(), + 0, + "the rewind stands for the rest of the pass" + ); + assert!(scanned.rewound); + } + + /// The record of a private request rebuilds byte for byte from the same + /// entry, and another entry's record encrypts under other IVs. + #[test] + fn should_record_a_private_request_the_same_way_every_time() { + let request = PrivateContactRequest { + account_reference: 7, + extended_public_key: [0x33; COMPACT_XPUB_LEN], + account_label: Some("label".to_string()), + auto_accept_proof: None, + shielded_address: None, + }; + let record = |commitment: [u8; 32]| { + private_request_record( + Identifier::from(ALICE), + Identifier::from(BOB), + 1, + 2, + &request, + &commitment, + &[9; 32], + BLOCK_TIME_MS, + ) + }; + assert_eq!(record([1; 32]), record([1; 32])); + assert!(record([1; 32]).came_privately()); + assert_ne!( + record([1; 32]).encrypted_public_key, + record([2; 32]).encrypted_public_key, + "another entry encrypts under other IVs" + ); + assert_eq!( + decrypt_extended_public_key(&[9; 32], &record([1; 32]).encrypted_public_key) + .expect("the xpub opens"), + vec![0x33; COMPACT_XPUB_LEN] + ); + } +} diff --git a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/contact_requests.rs b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/contact_requests.rs index b16b60ca532..d2e268f88c7 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/contact_requests.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/contact_requests.rs @@ -5,15 +5,33 @@ //! - Receiving contact requests (incoming) //! - Automatically establishing contacts when both parties send requests +use super::dashpay::PendingPrivateReply; use super::ManagedIdentity; use crate::changeset::{ - ContactChangeSet, ContactRequestEntry, ReceivedContactRequestKey, SentContactRequestKey, + ContactChangeSet, ContactRequestEntry, PrivateContactRequestsPosition, + ReceivedContactRequestKey, SentContactRequestKey, }; use crate::wallet::identity::crypto::contact_info::ContactInfoPrivateData; use crate::wallet::persister::WalletPersister; use crate::{ContactRequest, EstablishedContact}; +use dpp::identity::accessors::IdentityGettersV0; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::{KeyType, Purpose}; use dpp::prelude::Identifier; use platform_encryption::account_reference_version; +use sha2::{Digest, Sha256}; +use std::sync::atomic::{AtomicU64, Ordering}; + +/// The attempt id the next private request reservation or public request +/// claim takes. Process-wide, so a handle that outlived its wallet's removal +/// and re-creation under the same id cannot release a reservation or a claim +/// the new wallet made. +static NEXT_CONTACT_REQUEST_ATTEMPT: AtomicU64 = AtomicU64::new(0); + +/// A new attempt id ([`NEXT_CONTACT_REQUEST_ATTEMPT`]). +fn next_contact_request_attempt() -> u64 { + NEXT_CONTACT_REQUEST_ATTEMPT.fetch_add(1, Ordering::Relaxed) +} impl ManagedIdentity { /// The masked `accountReference` of the most recent request WE sent @@ -376,6 +394,12 @@ impl ManagedIdentity { // Rewind the receive cursor so the next sweep re-fetches the // now-un-ignored sender's on-chain requests. self.dashpay.high_water_received_ms = None; + self.dashpay.private_contact_requests_position = 0; + // A pass that read the store to its end passed over the sender's + // private requests: the store is not read for this identity until a + // pass reads it again from the rewound position, so a send reads it + // first. + self.dashpay.private_contact_requests_read = false; let mut cs = ContactChangeSet::default(); cs.unignored.insert((owner_id, *sender_id)); @@ -791,6 +815,339 @@ impl ManagedIdentity { advance_if_unchanged(self.dashpay.high_water_received_ms, snapshot, max_fetched); } + /// Note that a private sync read DashPay's private contact request store + /// to its end for this identity, with nothing held back. + pub(crate) fn note_private_contact_requests_read(&mut self) { + self.dashpay.private_contact_requests_read = true; + } + + /// Advance the private contact request cursor to `next_position` once + /// every entry before it was read and what it held for this identity + /// ingested. Never moves it back, and leaves it where a concurrent + /// [`Self::unignore_sender`] rewound it from `snapshot`. + pub(crate) fn advance_private_contact_requests_position( + &mut self, + snapshot: u64, + next_position: u64, + ) { + let position = &mut self.dashpay.private_contact_requests_position; + if *position == snapshot && next_position > snapshot { + *position = next_position; + } + } + + /// Restore the position of DashPay's private contact request store this + /// identity reads from next, as a backend persisted it + /// (`ContactChangeSet::private_contact_requests_positions`): the loader's + /// replay, for a backend attesting + /// [`PRIVATE_CONTACT_REQUEST_SCAN_STATE`](crate::changeset::PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE). + /// The record replayed last decides the identity's load-time hold, so a + /// later record cleared lifts an earlier marked one; the live + /// per-recipient reservations are left alone, and a restoration from + /// the shielded store's journal holds every pair whose payment may still + /// execute whatever the marker says. + pub fn apply_private_contact_requests_position( + &mut self, + position: PrivateContactRequestsPosition, + ) { + self.dashpay.private_contact_requests_position = position.position; + self.dashpay.private_scan_keys = Some(position.key_fingerprint); + self.dashpay.private_payments_unrestored = position.private_payments_unresolved; + } + + /// The reservations of this identity's private payments from an earlier + /// process are restored from the shielded store: release the hold a load + /// set ([`DashPayState::private_payments_unrestored`]). Returns whether it + /// was set, so the caller stores the record it changed. + /// + /// [`DashPayState::private_payments_unrestored`]: super::DashPayState::private_payments_unrestored + #[cfg(feature = "shielded")] + pub(crate) fn note_private_payments_restored(&mut self) -> bool { + std::mem::take(&mut self.dashpay.private_payments_unrestored) + } + + /// The fingerprint of the keys a private sync scans DashPay's private + /// contact request store with for this identity: SHA-256 over its HD + /// index and each `ECDSA_SECP256K1` `ENCRYPTION` or `DECRYPTION` key, by + /// id, purpose and data, in key id order. Disabled keys count, as an + /// entry sealed before a key was disabled still opens with it. + pub fn private_scan_key_fingerprint(&self) -> [u8; 32] { + let mut hasher = Sha256::new(); + match self.identity_index { + Some(index) => { + hasher.update([1]); + hasher.update(index.to_le_bytes()); + } + None => hasher.update([0]), + } + for (key_id, key) in self.identity.public_keys() { + if key.key_type() != KeyType::ECDSA_SECP256K1 + || !matches!(key.purpose(), Purpose::ENCRYPTION | Purpose::DECRYPTION) + { + continue; + } + let data = key.data().as_slice(); + hasher.update(key_id.to_le_bytes()); + hasher.update([key.purpose() as u8]); + hasher.update((data.len() as u64).to_le_bytes()); + hasher.update(data); + } + hasher.finalize().into() + } + + /// Tie the private read position and the read-to-the-end flag to the + /// identity's scanning keys: when the keys no longer match the ones they + /// hold for, entries to or from a key added since were never opened, so + /// both are rewound to read the store again from 0. Returns the position + /// record to store when it changed. + pub(crate) fn reconcile_private_scan_keys(&mut self) -> Option { + let key_fingerprint = self.private_scan_key_fingerprint(); + if self.dashpay.private_scan_keys == Some(key_fingerprint) { + return None; + } + if self.dashpay.private_scan_keys.is_some() + || self.dashpay.private_contact_requests_position > 0 + { + self.dashpay.private_contact_requests_position = 0; + self.dashpay.private_contact_requests_read = false; + } + self.dashpay.private_scan_keys = Some(key_fingerprint); + Some(self.private_contact_requests_position_record()) + } + + /// The private read position as a backend stores it: with the + /// fingerprint of the keys it holds for, and whether a private payment of + /// this identity may be unresolved + /// ([`DashPayState::private_payments_unresolved`]). + /// + /// [`DashPayState::private_payments_unresolved`]: super::DashPayState::private_payments_unresolved + pub(crate) fn private_contact_requests_position_record( + &self, + ) -> PrivateContactRequestsPosition { + PrivateContactRequestsPosition { + position: self.dashpay.private_contact_requests_position, + key_fingerprint: self + .dashpay + .private_scan_keys + .unwrap_or_else(|| self.private_scan_key_fingerprint()), + private_payments_unresolved: self.dashpay.private_payments_unresolved(), + } + } + + /// Restore the shielded address `contact`'s tracked private request to + /// this identity carries, as a backend persisted it + /// (`ContactChangeSet::private_shielded_addresses`): the loader's replay. + pub fn apply_private_shielded_address(&mut self, contact: Identifier, address: [u8; 43]) { + self.dashpay + .private_shielded_addresses + .insert(contact, address); + } + + /// Replay a persisted removal of `contact`'s shielded address + /// (`ContactChangeSet::removed_private_shielded_addresses`). + pub fn apply_removed_private_shielded_address(&mut self, contact: &Identifier) { + self.dashpay.private_shielded_addresses.remove(contact); + } + + /// Drop the private contact request scan state a load restored, for a + /// backend that does not attest it persists it: the store is read from + /// its start, and the addresses come back with the requests that carry + /// them. A restored hold on an unresolved private payment + /// ([`DashPayState::private_payments_unrestored`]) is kept: a backend that + /// handed one back kept it when it attested the capability, and dropping + /// it would let the identity publish the pair while the payment may still + /// execute. Kept, it holds until a restoration clears it. + /// + /// [`DashPayState::private_payments_unrestored`]: super::DashPayState::private_payments_unrestored + pub(crate) fn forget_private_contact_scan_state(&mut self) { + self.dashpay.private_contact_requests_position = 0; + self.dashpay.private_scan_keys = None; + self.dashpay.private_shielded_addresses.clear(); + } + + /// Record the shielded address `request`, a private request from + /// `request.sender_id`, carries, when it is the request tracked from that + /// contact: an older request ingest passed over does not replace the + /// address of the newer one. + pub(crate) fn note_private_shielded_address( + &mut self, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + ) { + let tracked = self + .dashpay + .incoming_request(&request.sender_id) + .is_some_and(|tracked| { + tracked.came_privately() + && tracked.account_reference == request.account_reference + && tracked.encrypted_public_key == request.encrypted_public_key + }); + if !tracked { + return; + } + match shielded_address { + Some(address) => { + self.dashpay + .private_shielded_addresses + .insert(request.sender_id, address); + } + None => { + self.dashpay + .private_shielded_addresses + .remove(&request.sender_id); + } + } + } + + /// Reserve the one private request to `recipient` that may be in flight, + /// returning the attempt that holds it; `None` when one already is. See + /// [`DashPayState::private_reply_pending`](super::DashPayState::private_reply_pending). + pub(crate) fn reserve_private_reply(&mut self, recipient: Identifier) -> Option { + // A public request being sent to the recipient holds the pair too. + if self + .dashpay + .pending_private_replies + .contains_key(&recipient) + || self + .dashpay + .public_contact_request_claims + .contains_key(&recipient) + { + return None; + } + Some(self.insert_private_reply(recipient, None)) + } + + /// Claim the pair with `recipient` for a public request about to be + /// sent, under a new attempt it returns: until that attempt releases it + /// ([`Self::release_public_contact_request`]), no private request to the + /// recipient is reserved. Concurrent public sends each hold their own. + pub(crate) fn claim_public_contact_request(&mut self, recipient: Identifier) -> u64 { + let attempt = next_contact_request_attempt(); + self.dashpay + .public_contact_request_claims + .entry(recipient) + .or_default() + .insert(attempt); + attempt + } + + /// Release the claim `attempt` took ([`Self::claim_public_contact_request`]); + /// another attempt's claim on the pair stays. + pub(crate) fn release_public_contact_request(&mut self, recipient: &Identifier, attempt: u64) { + if let Some(claims) = self + .dashpay + .public_contact_request_claims + .get_mut(recipient) + { + claims.remove(&attempt); + if claims.is_empty() { + self.dashpay.public_contact_request_claims.remove(recipient); + } + } + } + + /// Reserve `recipient` under a new attempt, `commitment` its entry's. + fn insert_private_reply(&mut self, recipient: Identifier, commitment: Option<[u8; 32]>) -> u64 { + let attempt = next_contact_request_attempt(); + self.dashpay.pending_private_replies.insert( + recipient, + PendingPrivateReply { + attempt, + commitment, + }, + ); + attempt + } + + /// Note the commitment of the entry `attempt` sealed its request to + /// `recipient` as, which the private sync settles it by. Nothing when the + /// recipient is not reserved by that attempt. + pub(crate) fn note_private_reply_commitment( + &mut self, + recipient: Identifier, + attempt: u64, + commitment: [u8; 32], + ) { + if let Some(pending) = self + .dashpay + .pending_private_replies + .get_mut(&recipient) + .filter(|pending| pending.attempt == attempt) + { + pending.commitment = Some(commitment); + } + } + + /// Drop the reservation `attempt` holds on `recipient`: its request was + /// never paid for, or its payment definitely failed. Nothing when another + /// attempt holds the recipient by now. + pub(crate) fn release_private_reply(&mut self, recipient: &Identifier, attempt: u64) { + if self + .dashpay + .pending_private_replies + .get(recipient) + .is_some_and(|pending| pending.attempt == attempt) + { + self.dashpay.pending_private_replies.remove(recipient); + } + } + + /// Release every pending request sealed as one of `commitments`, whose + /// payments' journal records were discharged (read back elsewhere, never + /// delivered, or proven expired), whichever attempt holds it. + pub(crate) fn release_discharged_private_replies( + &mut self, + commitments: &std::collections::BTreeSet<[u8; 32]>, + ) { + self.dashpay.pending_private_replies.retain(|_, pending| { + pending + .commitment + .is_none_or(|commitment| !commitments.contains(&commitment)) + }); + } + + /// Settle the pending request to `recipient` sealed as `commitment`: the + /// private sync read that entry back and recorded it. + pub(crate) fn settle_private_reply(&mut self, recipient: &Identifier, commitment: &[u8; 32]) { + if self + .dashpay + .pending_private_replies + .get(recipient) + .is_some_and(|pending| pending.commitment == Some(*commitment)) + { + self.dashpay.pending_private_replies.remove(recipient); + } + } + + /// Reserve `recipient` again for the entry sealed as `commitment`, whose + /// payment is unconfirmed since before a restart. A recipient already + /// reserved keeps its attempt, and gets the commitment only when it has + /// none yet. + pub(crate) fn restore_private_reply(&mut self, recipient: Identifier, commitment: [u8; 32]) { + match self.dashpay.pending_private_replies.get_mut(&recipient) { + Some(pending) => { + pending.commitment.get_or_insert(commitment); + } + None => { + self.insert_private_reply(recipient, Some(commitment)); + } + } + } + + /// Hold `recipient` for the entry sealed as `commitment`, whose payment is + /// unconfirmed since before a restart, instead of the one the attempt + /// reserving it sealed: that one is not paid for, so only this entry can + /// settle the reservation. + pub(crate) fn retarget_private_reply(&mut self, recipient: Identifier, commitment: [u8; 32]) { + match self.dashpay.pending_private_replies.get_mut(&recipient) { + Some(pending) => pending.commitment = Some(commitment), + None => { + self.insert_private_reply(recipient, Some(commitment)); + } + } + } + /// Sent-direction counterpart of [`Self::advance_high_water_received`]; /// same compare-and-advance semantics and caller contract. pub fn advance_high_water_sent(&mut self, snapshot: Option, max_fetched: Option) { @@ -946,6 +1303,26 @@ mod tests { WalletPersister::new([0u8; 32], Arc::new(Failing)) } + /// The private scan state replayed last decides an identity's load-time + /// hold: a later record cleared lifts an earlier marked one. + #[test] + fn should_take_the_last_replayed_private_payment_marker() { + let mut managed = create_test_identity([0x31; 32]); + let marked = |unresolved: bool| PrivateContactRequestsPosition { + private_payments_unresolved: unresolved, + ..managed_position() + }; + managed.apply_private_contact_requests_position(marked(true)); + assert!(managed.dashpay().private_payments_unrestored()); + managed.apply_private_contact_requests_position(marked(false)); + assert!(!managed.dashpay().private_payments_unrestored()); + } + + /// A private scan state record with nothing marked. + fn managed_position() -> PrivateContactRequestsPosition { + create_test_identity([0x32; 32]).private_contact_requests_position_record() + } + fn create_test_identity(id_bytes: [u8; 32]) -> ManagedIdentity { let identity_v0 = IdentityV0 { id: Identifier::from(id_bytes), @@ -2140,4 +2517,74 @@ mod tests { "replay un-ignore must not rewind the live cursor" ); } + + /// A reservation belongs to the attempt that made it: once attempt A's + /// request is read back and settled and B reserves the recipient, A's + /// late cleanup neither releases B's reservation nor notes a commitment + /// on it. + #[test] + fn should_keep_a_later_attempts_reservation_from_an_earlier_attempts_cleanup() { + let mut managed = create_test_identity([1u8; 32]); + let bob = Identifier::from([2u8; 32]); + + let a = managed.reserve_private_reply(bob).expect("A reserves"); + assert_eq!(managed.reserve_private_reply(bob), None, "one at a time"); + managed.note_private_reply_commitment(bob, a, [0xA; 32]); + managed.settle_private_reply(&bob, &[0xA; 32]); + assert!(!managed.dashpay().private_reply_pending(&bob)); + + let b = managed.reserve_private_reply(bob).expect("B reserves"); + assert_ne!(a, b); + managed.release_private_reply(&bob, a); + managed.note_private_reply_commitment(bob, a, [0xA; 32]); + assert!( + managed.dashpay().private_reply_pending(&bob), + "A's cleanup leaves B's reservation" + ); + managed.settle_private_reply(&bob, &[0xA; 32]); + assert!( + managed.dashpay().private_reply_pending(&bob), + "A's entry does not settle B's request" + ); + + managed.note_private_reply_commitment(bob, b, [0xB; 32]); + managed.settle_private_reply(&bob, &[0xB; 32]); + assert!(!managed.dashpay().private_reply_pending(&bob)); + } + + /// A reservation restored from an unconfirmed payment takes the entry's + /// commitment; one already held keeps its attempt and gets a commitment + /// only when it has none, unless it is retargeted to the restored entry. + #[test] + fn should_restore_a_reservation_without_taking_over_a_held_one() { + let mut managed = create_test_identity([1u8; 32]); + let bob = Identifier::from([2u8; 32]); + let carol = Identifier::from([3u8; 32]); + + managed.restore_private_reply(bob, [0xB; 32]); + assert!(managed.dashpay().private_reply_pending(&bob)); + assert_eq!(managed.reserve_private_reply(bob), None); + managed.settle_private_reply(&bob, &[0xB; 32]); + assert!(!managed.dashpay().private_reply_pending(&bob)); + + let held = managed.reserve_private_reply(carol).expect("Carol is free"); + managed.restore_private_reply(carol, [0xC; 32]); + managed.restore_private_reply(carol, [0xD; 32]); + managed.release_private_reply(&carol, held + 1); + assert!( + managed.dashpay().private_reply_pending(&carol), + "the attempt holding it is kept" + ); + managed.settle_private_reply(&carol, &[0xD; 32]); + assert!( + managed.dashpay().private_reply_pending(&carol), + "only a missing commitment is filled in" + ); + + managed.retarget_private_reply(carol, [0xD; 32]); + managed.settle_private_reply(&carol, &[0xC; 32]); + assert!(managed.dashpay().private_reply_pending(&carol)); + managed.settle_private_reply(&carol, &[0xD; 32]); + assert!(!managed.dashpay().private_reply_pending(&carol)); + } } diff --git a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/dashpay.rs b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/dashpay.rs index dbade254b1b..acd334985c4 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/dashpay.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/dashpay.rs @@ -13,6 +13,31 @@ use crate::wallet::identity::{ use dpp::prelude::Identifier; use std::collections::{BTreeMap, BTreeSet}; +/// How an identity and a contact exchanged private contact requests (protocol +/// version 15), and what those carry beyond what a [`ContactRequest`] holds. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct PrivateContact { + /// The contact's tracked request to this identity came privately + /// ([`ContactRequest::came_privately`]), so accepting it replies privately + /// too. + pub received_privately: bool, + /// This identity's tracked request to the contact went privately. + pub sent_privately: bool, + /// The Orchard address the contact asked to be paid at privately: an + /// 11-byte diversifier and a 32-byte transmission key. + pub shielded_address: Option<[u8; 43]>, +} + +/// The private request to one recipient that may be in flight: the attempt +/// that reserved it and, once sealed, the commitment of its entry. Only that +/// attempt notes the commitment or releases it; the private sync settles it +/// by the commitment. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct PendingPrivateReply { + pub(crate) attempt: u64, + pub(crate) commitment: Option<[u8; 32]>, +} + /// DashPay social state carried by one [`ManagedIdentity`](super::ManagedIdentity). #[derive(Debug, Clone, Default)] pub struct DashPayState { @@ -79,6 +104,69 @@ pub struct DashPayState { /// High-water mark for the sent direction (`$ownerId == me`). pub(super) high_water_sent_ms: Option, + /// The position of DashPay's private contact request store (protocol + /// version 15) to scan from next for this identity: every entry before it + /// was read and what it held for this identity ingested. Persisted, after + /// what the entries before it held, for a backend attesting + /// `PRIVATE_CONTACT_REQUEST_SCAN_STATE`, so a relaunch reads on from it; + /// otherwise a cold start scans the store again from 0, and ingest is a + /// fixpoint. [`ManagedIdentity::unignore_sender`] rewinds it, as it does + /// the received cursor. + /// + /// [`ManagedIdentity::unignore_sender`]: super::ManagedIdentity::unignore_sender + pub(super) private_contact_requests_position: u64, + + /// Whether a private sync read DashPay's private contact request store + /// to its end for this identity since the wallet was loaded, with nothing + /// held back. Until then a private send or accept reads it first: a + /// request paid for in an earlier process may be in the store unread, and + /// the reservation that kept it from being paid twice did not survive. + /// In memory only; a cursor rewind leaves it set. + pub(super) private_contact_requests_read: bool, + + /// The fingerprint of the scanning keys `private_contact_requests_position` + /// and `private_contact_requests_read` hold for + /// (`ManagedIdentity::private_scan_key_fingerprint`): entries to or from + /// a key added since were not opened. When the identity's keys no + /// longer match it, both are rewound (`reconcile_private_scan_keys`). + /// `None` before the first reconcile. + pub(super) private_scan_keys: Option<[u8; 32]>, + + /// The shielded address each contact's tracked private request to this + /// identity carries. Persisted for a backend attesting + /// `PRIVATE_CONTACT_REQUEST_SCAN_STATE`; otherwise rebuilt by the scan a + /// cold start runs. Whether a request came privately is the request's + /// own ([`ContactRequest::came_privately`]), so it holds across restarts. + pub(super) private_shielded_addresses: BTreeMap, + + /// The private requests to each recipient that are being paid for, or + /// were paid for and are not yet read back from the store, each owned by + /// the attempt that reserved it. While one is pending no other request to + /// the recipient is sent, private or public, as the store keeps every + /// entry appended and a second payment could execute too. The private + /// sync settles one when it reads the entry back and records it. In + /// memory only: after a restart, those whose payment is unconfirmed are + /// restored from the shielded store's unconfirmed spends. + pub(super) pending_private_replies: BTreeMap, + + /// The public requests to each recipient being sent, by attempt: each + /// claims the pair from before its channel is decided until it is + /// submitted and recorded, and while one does no private request to the + /// recipient is reserved, as a reserved one refuses a public send. A + /// claim is released only by its own attempt, whose id is process-wide, + /// so a send that outlived its wallet's removal and re-creation under the + /// same id cannot release a claim of the new wallet. In memory only. + pub(super) public_contact_request_claims: BTreeMap>, + + /// Whether this identity was loaded with a private contact request + /// payment that may be unresolved + /// (`PrivateContactRequestsPosition::private_payments_unresolved`) and no + /// restoration from the shielded store has restored its reservations + /// since. While set, the identity sends and accepts no public contact + /// request, as no reservation holds the pair its payment is for. In + /// memory, from what a load restored. + pub(super) private_payments_unrestored: bool, + /// Lowest Platform-assigned `$createdAtCoreBlockHeight` among OUR sent /// `contactRequest` docs to each recipient seen by a sweep this process. /// @@ -233,6 +321,89 @@ impl DashPayState { .or_else(|| self.sent_contact_requests.get(contact)) } + /// The contact's tracked request to us: the established contact's + /// incoming side, else a pending incoming request. + pub fn incoming_request(&self, contact: &Identifier) -> Option<&ContactRequest> { + self.established_contacts + .get(contact) + .map(|established| &established.incoming_request) + .or_else(|| self.incoming_contact_requests.get(contact)) + } + + /// The position of DashPay's private contact request store to scan from + /// next for this identity; see the field doc. + pub fn private_contact_requests_position(&self) -> u64 { + self.private_contact_requests_position + } + + /// Whether a private request to `contact` is being paid for, or was paid + /// for and is not yet read back from the store; see the field doc. + pub fn private_reply_pending(&self, contact: &Identifier) -> bool { + self.pending_private_replies.contains_key(contact) + } + + /// Whether any private request this identity sent is reserved as + /// pending ([`Self::private_reply_pending`]). + #[cfg(feature = "shielded")] + pub(crate) fn private_replies_pending(&self) -> bool { + !self.pending_private_replies.is_empty() + } + + /// Whether this identity was loaded with a private contact request payment + /// that may be unresolved, and no restoration from the shielded store has + /// restored its reservations since: its public contact requests are held. + pub fn private_payments_unrestored(&self) -> bool { + self.private_payments_unrestored + } + + /// Whether a private contact request this identity paid for may still be + /// unresolved, as its position record stores it + /// (`PrivateContactRequestsPosition::private_payments_unresolved`): it + /// holds a private request it is paying for or has not read back, or it + /// was loaded with one no restoration has restored since. + pub fn private_payments_unresolved(&self) -> bool { + !self.pending_private_replies.is_empty() || self.private_payments_unrestored + } + + /// The shielded address kept for `contact`, whichever way the tracked + /// requests went; [`Self::private_contact`] gives it only while the + /// contact's request to us came privately. + pub(crate) fn private_shielded_address(&self, contact: &Identifier) -> Option<[u8; 43]> { + self.private_shielded_addresses.get(contact).copied() + } + + /// Whether a public request to `contact` is being sent; see the field + /// doc. + pub fn public_contact_request_claimed(&self, contact: &Identifier) -> bool { + self.public_contact_request_claims.contains_key(contact) + } + + /// Whether a private sync read DashPay's private contact request store + /// to its end for this identity since load; see the field doc. + pub fn private_contact_requests_read(&self) -> bool { + self.private_contact_requests_read + } + + /// How this identity and `contact` exchanged private requests, from the + /// tracked requests either way; `None` when neither went privately. + pub fn private_contact(&self, contact: &Identifier) -> Option { + let received_privately = self + .incoming_request(contact) + .is_some_and(ContactRequest::came_privately); + let sent_privately = self + .outgoing_request(contact) + .is_some_and(ContactRequest::came_privately); + (received_privately || sent_privately).then(|| PrivateContact { + received_privately, + sent_privately, + shielded_address: self + .private_shielded_addresses + .get(contact) + .copied() + .filter(|_| received_privately), + }) + } + /// Whether a sent-request sweep completed this process; see the field doc. pub fn sent_sweep_completed(&self) -> bool { self.sent_sweep_completed diff --git a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/mod.rs b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/mod.rs index 436dbaaabf6..c11f1e91f8a 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/mod.rs @@ -9,7 +9,7 @@ mod dashpay; mod identity_ops; mod sync; -pub use dashpay::DashPayState; +pub use dashpay::{DashPayState, PrivateContact}; // `block_time` + `key_storage` moved to `crate::wallet::identity::types`. // Re-export so every `impl ManagedIdentity` block below keeps working diff --git a/packages/rs-platform-wallet/src/wallet/identity/state/mod.rs b/packages/rs-platform-wallet/src/wallet/identity/state/mod.rs index affa70d0156..05f321ec6ef 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/state/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/state/mod.rs @@ -8,7 +8,7 @@ pub mod managed_identity; pub mod manager; -pub use managed_identity::{BlockTime, DashPayState, ManagedIdentity}; +pub use managed_identity::{BlockTime, DashPayState, ManagedIdentity, PrivateContact}; pub use manager::IdentityLocation; pub use manager::IdentityManager; pub use manager::RegistrationIndex; diff --git a/packages/rs-platform-wallet/src/wallet/identity/types/dashpay/contact_request.rs b/packages/rs-platform-wallet/src/wallet/identity/types/dashpay/contact_request.rs index c039823fa34..d6ac8b83ec7 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/types/dashpay/contact_request.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/types/dashpay/contact_request.rs @@ -70,6 +70,15 @@ impl ContactRequest { } } + /// Whether the request came as a private contact request (protocol + /// version 15) rather than a `contactRequest` document. A document always + /// carries its Platform-assigned `$createdAtCoreBlockHeight`, which + /// DashPay requires and no Core chain puts at 0; a private request's entry + /// carries none, so the wallet records it at 0. + pub fn came_privately(&self) -> bool { + self.core_height_created_at == 0 + } + /// Check if this is an outgoing request for the given identity pub fn is_outgoing(&self, identity_id: &Identifier) -> bool { self.sender_id == *identity_id diff --git a/packages/rs-platform-wallet/src/wallet/mod.rs b/packages/rs-platform-wallet/src/wallet/mod.rs index 748f17de197..bbf1c58caf4 100644 --- a/packages/rs-platform-wallet/src/wallet/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/mod.rs @@ -26,11 +26,11 @@ pub use platform_addresses::{ PerAccountPlatformAddressState, PerWalletPlatformAddressState, PlatformAddressTag, PlatformAddressWallet, }; -#[cfg(feature = "shielded")] -pub use platform_wallet::ShieldedShieldPreflight; pub use platform_wallet::{ PlatformWallet, PlatformWalletInfo, WalletId, WalletStateReadGuard, WalletStateWriteGuard, }; +#[cfg(feature = "shielded")] +pub use platform_wallet::{PrivateReplyPayment, ShieldedShieldPreflight}; pub use provider_key_at_index::{ProviderDerivedKey, ProviderKeyKind}; #[cfg(feature = "shielded")] pub use shielded::operations::shield_fee_reserve_credits; diff --git a/packages/rs-platform-wallet/src/wallet/persister.rs b/packages/rs-platform-wallet/src/wallet/persister.rs index a61e9082e17..f2f3ba3297b 100644 --- a/packages/rs-platform-wallet/src/wallet/persister.rs +++ b/packages/rs-platform-wallet/src/wallet/persister.rs @@ -4,6 +4,8 @@ //! - [`WalletPersister`] — wraps the shared [`PlatformWalletPersistence`] with //! a fixed `wallet_id` so callers don't need to pass the ID on every call. +use std::collections::BTreeSet; +use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; use dashcore::Txid; @@ -57,6 +59,47 @@ pub struct WalletPersister { wallet_id: WalletId, inner: Arc, contact_payment_gate: Arc>, + /// Commitments of private contact request entries a private sync read + /// back and recorded durably, whose records in the shielded store's + /// private payment journal are to be discharged: the wallet's view + /// notes them, and whoever holds the store discharges them (the shielded + /// coordinator's sync pass, a restoration, a private payment). Shared by + /// the clones the wallet and the coordinator hold. In memory only: a + /// record left undischarged by a restart holds nothing, as the request + /// it pays for is recorded. + read_back_private_entries: Arc>>, + /// The (sender, recipient) pairs of journaled private payments whose + /// sender the wallet did not manage when they were last restored: an + /// identity added since holds them until the next restoration reserves + /// them. Replaced by every restoration. In memory only. + absent_sender_private_pairs: Arc>>, + /// The commitments of the private contact request payments whose journal + /// records were discharged in this session (read back, never delivered, + /// or proven expired). Shared by the clones the wallet and the + /// coordinator hold. In memory only, as are the reservations they guard. + discharged_private_entries: Arc>, + /// Whether the reservations of this wallet's unconfirmed private contact + /// request payments from an earlier process may still be unrestored: + /// raised at load for a wallet that bound a shielded account before, and + /// lowered once a restoration from the shielded store succeeds. While it + /// is up, public contact requests are neither sent nor accepted. Shared by + /// the clones the wallet and its views hold. In memory only. + private_payments_unrestored: Arc, +} + +/// The commitments [`WalletPersister::note_discharged_private_entries`] noted, +/// both sets updated under one lock. +#[derive(Default)] +struct DischargedPrivateEntries { + /// Noted since the last take, for the wallet to release the reservations + /// it holds for them ([`WalletPersister::take_unreleased_discharged_private_entries`]). + unreleased: BTreeSet<[u8; 32]>, + /// Every one noted in this session, kept after the take: a restoration + /// that read a record before it was discharged, and reserves its pair + /// after, checks it here first under the wallet lock + /// ([`WalletPersister::private_entry_discharged`]). Only a shielded build + /// keeps a journal to discharge: empty otherwise. + session: BTreeSet<[u8; 32]>, } impl WalletPersister { @@ -65,9 +108,139 @@ impl WalletPersister { wallet_id, inner, contact_payment_gate: Arc::new(tokio::sync::Mutex::new(())), + read_back_private_entries: Arc::default(), + absent_sender_private_pairs: Arc::default(), + discharged_private_entries: Arc::default(), + private_payments_unrestored: Arc::default(), } } + /// Note the commitments of private contact request entries a private sync + /// read back and recorded durably, for their journal records to be + /// discharged ([`Self::take_read_back_private_entries`]). + pub(crate) fn note_read_back_private_entries( + &self, + commitments: impl IntoIterator, + ) { + self.read_back_entries().extend(commitments); + } + + /// Take the commitments [`Self::note_read_back_private_entries`] noted: + /// a caller that fails to discharge one notes it again. + #[cfg(feature = "shielded")] + pub(crate) fn take_read_back_private_entries(&self) -> BTreeSet<[u8; 32]> { + std::mem::take(&mut *self.read_back_entries()) + } + + /// Hold `pairs`, the (sender, recipient) pairs of journaled private + /// payments whose sender the wallet does not manage now, replacing the + /// ones an earlier restoration held. + #[cfg(feature = "shielded")] + pub(crate) fn hold_absent_sender_private_pairs( + &self, + pairs: BTreeSet<(Identifier, Identifier)>, + ) { + *self + .absent_sender_private_pairs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) = pairs; + } + + /// Whether a journaled private payment from `sender` to `recipient` holds + /// the pair though the wallet did not manage `sender` when it was last + /// restored ([`Self::hold_absent_sender_private_pairs`]). + pub(crate) fn absent_sender_private_pair_held( + &self, + sender: &Identifier, + recipient: &Identifier, + ) -> bool { + self.absent_sender_private_pairs + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .contains(&(*sender, *recipient)) + } + + /// Note `commitments`, whose private payments' journal records were just + /// discharged: a restoration that read them before skips them + /// ([`Self::private_entry_discharged`]), and the wallet releases what it + /// still reserves for them + /// ([`Self::take_unreleased_discharged_private_entries`]). + #[cfg(feature = "shielded")] + pub(crate) fn note_discharged_private_entries( + &self, + commitments: impl IntoIterator, + ) { + let mut discharged = self.discharged_entries(); + for commitment in commitments { + discharged.unreleased.insert(commitment); + discharged.session.insert(commitment); + } + } + + /// Whether the journal record of `commitment` was discharged in this + /// session, its reservation released or not. + pub(crate) fn private_entry_discharged(&self, commitment: &[u8; 32]) -> bool { + self.discharged_entries().session.contains(commitment) + } + + /// Whether [`Self::note_discharged_private_entries`] noted any commitment + /// since the last take. + pub(crate) fn has_unreleased_discharged_private_entries(&self) -> bool { + !self.discharged_entries().unreleased.is_empty() + } + + /// Take the commitments [`Self::note_discharged_private_entries`] noted + /// since the last take. Take them only where they are applied with + /// nothing awaited in between. + pub(crate) fn take_unreleased_discharged_private_entries(&self) -> BTreeSet<[u8; 32]> { + std::mem::take(&mut self.discharged_entries().unreleased) + } + + fn discharged_entries(&self) -> std::sync::MutexGuard<'_, DischargedPrivateEntries> { + self.discharged_private_entries + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + } + + /// Whether [`Self::note_read_back_private_entries`] noted any commitment + /// not taken yet. + #[cfg(feature = "shielded")] + pub(crate) fn has_read_back_private_entries(&self) -> bool { + !self.read_back_entries().is_empty() + } + + /// Hold this wallet's public contact requests, sent or accepted, until + /// [`Self::private_payments_restored`]: its shielded store may hold an + /// unconfirmed private payment from an earlier process whose pair no + /// reservation holds yet. + #[cfg(feature = "shielded")] + pub(crate) fn hold_until_private_payments_restored(&self) { + self.private_payments_unrestored + .store(true, Ordering::Release); + } + + /// The reservations of this wallet's unconfirmed private payments are + /// restored: release the hold [`Self::hold_until_private_payments_restored`] + /// raised. Called after the restored reservations are in place, so a send + /// that sees the hold gone sees them. + #[cfg(feature = "shielded")] + pub(crate) fn private_payments_restored(&self) { + self.private_payments_unrestored + .store(false, Ordering::Release); + } + + /// Whether [`Self::hold_until_private_payments_restored`] holds this + /// wallet's public contact requests. + pub(crate) fn private_payments_unrestored(&self) -> bool { + self.private_payments_unrestored.load(Ordering::Acquire) + } + + fn read_back_entries(&self) -> std::sync::MutexGuard<'_, BTreeSet<[u8; 32]>> { + self.read_back_private_entries + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + } + /// Keep contact-pool snapshots in reservation order across Core and /// withdrawal sends, including a Core broadcast rejection's pool rollback. /// @@ -136,6 +309,14 @@ impl WalletPersister { self.inner.persistence_capabilities() } + /// Whether the backend stores and restores the private contact request + /// sync's scan state + /// ([`PersistenceCapabilities::PRIVATE_CONTACT_REQUEST_SCAN_STATE`]). + pub(crate) fn persists_private_contact_scan_state(&self) -> bool { + self.persistence_capabilities() + .contains(PersistenceCapabilities::PRIVATE_CONTACT_SCAN_STATE_RESTART) + } + pub(crate) fn load(&self) -> Result { self.inner.load() } diff --git a/packages/rs-platform-wallet/src/wallet/platform_wallet.rs b/packages/rs-platform-wallet/src/wallet/platform_wallet.rs index 50a564b04df..a3ec4a49bbe 100644 --- a/packages/rs-platform-wallet/src/wallet/platform_wallet.rs +++ b/packages/rs-platform-wallet/src/wallet/platform_wallet.rs @@ -42,10 +42,46 @@ use dpp::fee::Credits; use dpp::identity::signer::Signer; use dpp::identity::{Identity, IdentityPublicKey}; use dpp::prelude::Identifier; +#[cfg(feature = "shielded")] +use { + super::identity::network::{ + dashpay_contract, restore_journaled_private_payments, JournaledPrivatePayment, + }, + super::shielded::operations::{ + discharge_read_back_private_payments, private_document_create_from_pool_with_metadata, + }, + super::shielded::store::{PendingRedrive, PrivateContactPayment, PrivatePaymentRecord}, + super::shielded::{NetworkShieldedCoordinator, OrchardKeySet, ShieldedStore}, + crate::{ + AutoAcceptProofSource, ContactCryptoProvider, ContactRequest, EstablishedContact, + PaymentStarted, + }, + dash_sdk::dapi_grpc::platform::v0::ResponseMetadata, + dash_sdk::platform::dashpay::PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + dpp::data_contract::accessors::v0::DataContractV0Getters, + dpp::platform_value::BinaryData, + dpp::serialization::PlatformDeserializableTrusted, + dpp::shielded::builder::OrchardProver, + dpp::state_transition::private_document_create_from_shielded_pool_transition::accessors::PrivateDocumentCreateFromShieldedPoolTransitionAccessorsV0, + dpp::state_transition::StateTransition, +}; /// Unique identifier for a wallet (32-byte hash). pub type WalletId = [u8; 32]; +/// What a private contact request reply is paid with, which +/// [`PlatformWallet::accept_contact_request_replying_in_kind`] asks for only +/// when the reply is private: the shielded coordinator, the seed the spend +/// authority is derived from (dropped once it is), the shielded account and +/// the prover. +#[cfg(feature = "shielded")] +pub struct PrivateReplyPayment { + pub coordinator: Arc, + pub seed: Seed, + pub account: u32, + pub prover: P, +} + /// Cached capacity snapshot for shielding a Platform Payment account. /// /// The figures are computed from the same lexicographic address ordering and @@ -737,6 +773,7 @@ impl PlatformWallet { persister: wallet_persister.clone(), sync_fault, durable_cursors, + generation: Arc::clone(&generation), broadcaster: dashpay_broadcaster, // DashPay write helper: forwards to the live SDK, erasing its // generic write signatures behind concrete by-value methods. @@ -984,6 +1021,11 @@ impl PlatformWallet { /// registration (which would leave sync decrypting under one key /// while addresses and spends use another), nor between the /// restore's registration check and its store write. + /// + /// Once installed, the reservations of the private contact requests the + /// store holds unconfirmed payments for are restored into this wallet's + /// identities ([`Self::restore_private_contact_payments`]), with no host + /// call and no crypto provider. #[cfg(feature = "shielded")] async fn install_shielded_views( &self, @@ -991,6 +1033,60 @@ impl PlatformWallet { coordinator: &Arc, start: crate::changeset::ClientStartState, snapshot_generation: u64, + ) -> Result<(), PlatformWalletError> { + self.install_shielded_views_in_one_transaction( + account_views, + coordinator, + start, + snapshot_generation, + ) + .await?; + // After the install transaction: the restore takes the store and + // wallet locks on its own. It never waits for the store, which a + // running scan may hold for minutes while a host binds on its UI + // thread: a busy store defers it to a task, the wallet's contact + // requests held until it succeeds. + let restore = self.private_payment_restore(coordinator); + match restore.restore_and_release(false).await { + Ok(Some(_)) => {} + Ok(None) => { + tracing::debug!( + wallet_id = %hex::encode(self.wallet_id), + "the shielded store is busy; private contact request payments are \ + restored once it frees" + ); + tokio::spawn(async move { + if let Err(error) = restore.restore_and_release(true).await { + tracing::warn!( + wallet_id = %hex::encode(restore.wallet_id), + %error, + "private contact request payments not restored; the wallet's \ + contact requests stay held" + ); + } + }); + } + Err(error) => { + tracing::warn!( + wallet_id = %hex::encode(self.wallet_id), + %error, + "private contact request payments not restored at bind; the wallet's \ + contact requests stay held until a restoration succeeds" + ); + } + } + Ok(()) + } + + /// The body of [`Self::install_shielded_views`]: one coordinator install + /// transaction. + #[cfg(feature = "shielded")] + async fn install_shielded_views_in_one_transaction( + &self, + account_views: std::collections::BTreeMap, + coordinator: &Arc, + start: crate::changeset::ClientStartState, + snapshot_generation: u64, ) -> Result<(), PlatformWalletError> { // Serialize registration with identity-shield proof construction. Otherwise // an account could disappear before that operation arms its durable guard. @@ -1529,6 +1625,399 @@ impl PlatformWallet { Ok(proven_balance) } + /// Send a DashPay private contact request (protocol version 15) from + /// `sender_identity_id` to `recipient_identity_id`, paying for it from + /// `account`'s shielded notes: nothing on chain names either identity. The + /// request carries `account_label`, `auto_accept_proof` and our + /// `shielded_address` for the contact, is signed by `signer` as the + /// sender's transitions are, and is sealed by `crypto`, which keeps the + /// encryption key. `seed` supplies the transient spend authority (see + /// [`shielded_transfer_to`](Self::shielded_transfer_to)). See + /// [`DashPayView::send_private_contact_request`](crate::DashPayView::send_private_contact_request). + #[cfg(feature = "shielded")] + #[allow(clippy::too_many_arguments)] + pub async fn send_private_contact_request( + &self, + coordinator: &Arc, + seed: &[u8], + account: u32, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + account_label: Option, + auto_accept_proof: AutoAcceptProofSource, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + prover: P, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + P: OrchardProver + Send + Sync, + { + let keyset = self.derive_spend_keyset(seed, account).await?; + self.identity + .dashpay() + .send_private_contact_request( + sender_identity_id, + recipient_identity_id, + account_label, + auto_accept_proof, + shielded_address, + signer, + crypto, + |entry, started| { + self.pay_private_contact_request( + coordinator, + keyset, + account, + sender_identity_id, + recipient_identity_id, + entry, + started, + prover, + ) + }, + ) + .await + } + + /// Accept a DashPay contact request that came privately, replying + /// privately and paying for the reply from `account`'s shielded notes, as + /// [`send_private_contact_request`](Self::send_private_contact_request) + /// pays and signing it with `signer`. `shielded_address` is the address + /// the contact may pay us at privately. See + /// [`DashPayView::accept_private_contact_request`](crate::DashPayView::accept_private_contact_request). + #[cfg(feature = "shielded")] + #[allow(clippy::too_many_arguments)] + pub async fn accept_private_contact_request( + &self, + coordinator: &Arc, + seed: &[u8], + account: u32, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + prover: P, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + P: OrchardProver + Send + Sync, + { + let keyset = self.derive_spend_keyset(seed, account).await?; + self.identity + .dashpay() + .accept_private_contact_request( + request, + shielded_address, + signer, + crypto, + |entry, started| { + self.pay_private_contact_request( + coordinator, + keyset, + account, + &request.recipient_id, + &request.sender_id, + entry, + started, + prover, + ) + }, + ) + .await + } + + /// Accept a DashPay contact request replying the way it came: privately, + /// paid from a shielded account, when it came privately and no reply + /// exists yet, else with a `contactRequest` document signed by `signer`. + /// `payment` gives what a private reply is paid with (the shielded + /// coordinator, the seed for the spend authority, the account and the + /// prover) and is called only for a private reply, so a public reply or + /// an adopted one needs neither shielded support nor the seed. See + /// [`DashPayView::accept_contact_request_replying_in_kind`](crate::DashPayView::accept_contact_request_replying_in_kind). + #[cfg(feature = "shielded")] + pub async fn accept_contact_request_replying_in_kind( + &self, + request: &ContactRequest, + shielded_address: Option<[u8; 43]>, + signer: &S, + crypto: &C, + payment: R, + ) -> Result + where + S: Signer + Send + Sync, + C: ContactCryptoProvider + Sync, + P: OrchardProver + Send + Sync, + Seed: AsRef<[u8]> + Send, + R: FnOnce() -> RFut + Send, + RFut: std::future::Future, PlatformWalletError>> + + Send, + { + self.identity + .dashpay() + .accept_contact_request_replying_in_kind( + request, + shielded_address, + signer, + crypto, + |entry, started| async move { + // Resolving the payment and deriving the spend key come + // first, before `started` is marked: a reply dropped + // meanwhile releases its reservation. + let PrivateReplyPayment { + coordinator, + seed, + account, + prover, + } = payment().await?; + let keyset = self.derive_spend_keyset(seed.as_ref(), account).await?; + drop(seed); + self.pay_private_contact_request( + &coordinator, + keyset, + account, + &request.recipient_id, + &request.sender_id, + entry, + started, + prover, + ) + .await + }, + ) + .await + } + + /// Restore, at startup, the pending DashPay private contact requests + /// whose payment is unconfirmed since before a restart, from the signed + /// transitions `coordinator`'s store keeps for this wallet's unconfirmed + /// spends: until a private sync reads each back, no other request to its + /// recipient is sent, privately or publicly. `crypto` opens the entries. + /// Returns the (sender, recipient) pairs restored. A private send runs + /// the same check right before it pays, so without this call a second + /// private request is still held; this call holds public ones too. A + /// payment saved with its request needs no call: binding restores it + /// ([`Self::restore_private_contact_payments`]). This call stays for + /// records saved before that, and for an identity added after the bind. + /// See + /// [`DashPayView::restore_pending_private_contact_requests`](crate::DashPayView::restore_pending_private_contact_requests) + /// for what it does not cover. + #[cfg(feature = "shielded")] + pub async fn restore_pending_private_contact_requests( + &self, + coordinator: &Arc, + crypto: &C, + ) -> Result, PlatformWalletError> + where + C: ContactCryptoProvider + Sync, + { + let entries = self + .unconfirmed_private_contact_requests(coordinator) + .await?; + self.identity + .dashpay() + .restore_pending_private_contact_requests(&entries, crypto) + .await + } + + /// Restore the reservations of this wallet's private contact request + /// payments that may still execute, from the shielded store's private + /// payment journal (sender, recipient, account reference and entry + /// commitment of each, saved durably before it could reach the network), + /// and release the holds a load set: no keys open anything. Run when the + /// wallet binds to the coordinator, so restored reservations hold every + /// public and private send from then on. Returns the (sender, recipient) + /// pairs restored. + /// + /// Fails closed, every hold staying and the wallet's raised if it was + /// not: for a coordinator of another network, a wallet removed or + /// re-created since this handle was taken, a journal record that does + /// not read back, a saved spend whose transition does not decode and + /// that no journal record accounts for, a private contact request + /// payment saved before the journal kept it with no request to restore + /// its pair from, and an identity loaded marking a private payment that + /// may be unresolved when the journal is new + /// ([`PlatformWalletError::ShieldedRecoveryCorrupted`]). A payment saved + /// before the journal kept it, with its request, is added to the + /// journal. + /// + /// Run also by [`PlatformWalletManager::configure_shielded`] for every + /// wallet the manager holds, and by its load for the wallets it loads once + /// the store is configured: no keys are needed. Once it succeeds, the + /// wallet's contact requests, which a load holds for a wallet that bound + /// a shielded account before, and for an identity whose position record + /// marks a private payment that may be unresolved + /// ([`PlatformWalletError::PrivateContactPaymentsNotRestored`]), go ahead. + /// + /// [`PlatformWalletManager::configure_shielded`]: crate::PlatformWalletManager::configure_shielded + #[cfg(feature = "shielded")] + pub async fn restore_private_contact_payments( + &self, + coordinator: &Arc, + ) -> Result, PlatformWalletError> { + self.private_payment_restore(coordinator) + .restore_and_release(true) + .await + .map(Option::unwrap_or_default) + } + + /// The handles a restoration of this wallet's private contact request + /// payments runs on ([`Self::restore_private_contact_payments`]). + #[cfg(feature = "shielded")] + fn private_payment_restore( + &self, + coordinator: &Arc, + ) -> PrivatePaymentRestore { + PrivatePaymentRestore { + coordinator: Arc::clone(coordinator), + wallet_manager: Arc::clone(&self.wallet_manager), + persister: self.persister.clone(), + generation: Arc::clone(&self.generation), + wallet_id: self.wallet_id, + network: self.sdk.network, + } + } + + /// Whether this wallet's public contact requests wait for its private + /// payments from an earlier process to be restored + /// ([`Self::restore_private_contact_payments`]): it bound a shielded account + /// before, or one of its identities was loaded marking a private payment + /// that may be unresolved. + pub async fn private_payments_held(&self) -> bool { + self.persister.private_payments_unrestored() + || self + .identity + .dashpay() + .any_private_payments_unrestored() + .await + } + + /// The entries of the private contact requests whose payment from this + /// wallet's notes `coordinator`'s store holds as unconfirmed. + #[cfg(feature = "shielded")] + async fn unconfirmed_private_contact_requests( + &self, + coordinator: &Arc, + ) -> Result, PlatformWalletError> { + let dashpay_id = dashpay_contract()?.id(); + let store = coordinator.store().read().await; + let redrives = store + .pending_redrives_for_wallet(self.wallet_id) + .map_err(PlatformWalletError::from)?; + Ok(private_contact_request_redrive_entries( + redrives.iter().map(|(_, redrive)| redrive), + dashpay_id, + )) + } + + /// Append the sealed private contact request `entry`, from + /// `sender_identity_id` to `recipient_identity_id`, to DashPay's store, + /// paying from `account`'s notes, and return the metadata of the proof + /// that it was appended. + /// + /// A request between the same pair whose payment is unconfirmed since + /// before a restart may still execute: then nothing is paid, and the + /// recipient stays reserved for that request + /// ([`PlatformWalletError::PrivateContactRequestPending`]). + /// + /// Holds the wallet generation's lifecycle gate from the liveness check + /// until the spend returns, so a removal waits for it. A wallet removed or + /// re-created since this handle was taken is refused with + /// [`PlatformWalletError::WalletRemovedBeforeSend`], with nothing + /// restored, recorded or paid. A caller must not hold the gate. + /// + /// `started` is marked right before the transition may reach the + /// network, after the restore check and the proof: until then a send + /// dropped releases its reservation. + #[cfg(feature = "shielded")] + #[allow(clippy::too_many_arguments)] + async fn pay_private_contact_request

( + &self, + coordinator: &Arc, + keyset: OrchardKeySet, + account: u32, + sender_identity_id: &Identifier, + recipient_identity_id: &Identifier, + entry: BinaryData, + started: PaymentStarted, + prover: P, + ) -> Result + where + P: OrchardProver, + { + // The generation's lifecycle gate, held across the liveness check, + // the restore and the spend, before the wallet manager lock as the + // teardown takes them: a removal cannot complete in between, so a + // removed wallet's spend never records activity, recreating its + // shielded state, or reaches the network. + let _lifecycle = self.core().generation_payment_guard().await; + if !self.core().is_current_generation().await { + return Err(PlatformWalletError::WalletRemovedBeforeSend(hex::encode( + self.wallet_id, + ))); + } + // An earlier payment of the pair the journal keeps may still execute, + // whether or not its redrive record reads: then nothing is paid. The + // journal is read strictly: a record that does not read refuses. + let journaled: Vec<(u32, [u8; 32])> = { + let mut store = coordinator.store().write().await; + discharge_read_back_private_payments(&mut *store, &self.persister, self.wallet_id); + store + .private_payment_journal(self.wallet_id) + .map_err(PlatformWalletError::from)? + .into_iter() + .filter(|record| { + Identifier::from(record.payment.sender) == *sender_identity_id + && Identifier::from(record.payment.recipient) == *recipient_identity_id + }) + .map(|record| (record.payment.account_reference, record.commitment)) + .collect() + }; + if self + .identity + .dashpay() + .hold_for_journaled_private_payment( + sender_identity_id, + recipient_identity_id, + &journaled, + ) + .await? + { + return Err(PlatformWalletError::PrivateContactRequestPending( + *recipient_identity_id, + )); + } + let dashpay_id = dashpay_contract()?.id(); + // Saved with the payment's record, so a restart restores the pair's + // reservation without opening the entry. + let private_contact = + started + .account_reference() + .map(|account_reference| PrivateContactPayment { + sender: sender_identity_id.to_buffer(), + recipient: recipient_identity_id.to_buffer(), + account_reference, + }); + private_document_create_from_pool_with_metadata( + &self.sdk, + coordinator.store(), + Some(&self.persister), + self.wallet_id, + &keyset, + account, + dashpay_id, + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + vec![entry], + private_contact, + &started, + &prover, + ) + .await + } + /// Withdraw from `account`'s notes to a Core L1 address /// (Base58Check string). `core_fee_per_byte` is the L1 fee /// rate (duffs/byte). `seed` supplies the transient spend @@ -2350,6 +2839,322 @@ impl DerefMut for WalletStateWriteGuard<'_> { } } +/// The commitments of the entries a saved private contact request payment +/// appends, read from its signed transition. Refused with +/// [`PlatformWalletError::ShieldedRecoveryCorrupted`] when the transition does +/// not decode, is not a private document create (type 29) appending to +/// DashPay's `privateContactRequest` store (`dashpay_id` being DashPay's +/// contract id), carries no entry, or carries one shorter than its 32-byte +/// commitment: the payment's pair is restored from it, so a payment that does +/// not read back fails the restoration rather than release the holds. +#[cfg(feature = "shielded")] +fn associated_payment_commitments( + redrive: &PendingRedrive, + dashpay_id: Identifier, +) -> Result, PlatformWalletError> { + let corrupted = |reason: String| PlatformWalletError::ShieldedRecoveryCorrupted { + account_index: None, + reason, + }; + let transition = match StateTransition::deserialize_from_bytes_trusted(&redrive.st_bytes) { + Ok(StateTransition::PrivateDocumentCreateFromShieldedPool(transition)) => transition, + Ok(_) => { + return Err(corrupted( + "a saved private contact request payment's transition is not a private \ + document create" + .to_string(), + )) + } + Err(error) => { + return Err(corrupted(format!( + "a saved private contact request payment's transition does not decode: {error}" + ))) + } + }; + if transition.data_contract_id() != dashpay_id + || transition.document_type_name() != PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME + { + return Err(corrupted(format!( + "a saved private contact request payment appends to {} {}, not DashPay's \ + {PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME}", + transition.data_contract_id(), + transition.document_type_name() + ))); + } + if transition.entries().is_empty() { + return Err(corrupted( + "a saved private contact request payment carries no entry".to_string(), + )); + } + transition + .entries() + .iter() + .map(|entry| { + entry + .as_slice() + .get(..32) + .and_then(|commitment| <[u8; 32]>::try_from(commitment).ok()) + .ok_or_else(|| { + corrupted( + "a saved private contact request payment's entry is shorter than its \ + commitment" + .to_string(), + ) + }) + }) + .collect() +} + +/// The handles a restoration of a wallet's private contact request payments +/// from the shielded store's journal runs on, owned, so one the store is +/// busy for runs on a task ([`PlatformWallet::restore_private_contact_payments`]). +#[cfg(feature = "shielded")] +struct PrivatePaymentRestore { + coordinator: Arc, + wallet_manager: Arc>>, + persister: WalletPersister, + generation: Arc, + wallet_id: WalletId, + network: dashcore::Network, +} + +#[cfg(feature = "shielded")] +impl PrivatePaymentRestore { + /// Restore, and raise or release the wallet's hold as it went: released + /// once a restoration succeeds, raised when it fails or cannot run yet. + /// `Ok(None)` when the store is busy and `wait` is false. + async fn restore_and_release( + &self, + wait: bool, + ) -> Result>, PlatformWalletError> { + let restored = self.restore(wait).await; + match &restored { + Ok(Some(_)) => self.persister.private_payments_restored(), + Ok(None) | Err(_) => self.persister.hold_until_private_payments_restored(), + } + restored + } + + /// The restoration [`PlatformWallet::restore_private_contact_payments`] + /// documents, fenced against the wallet's replacement: the generation's + /// lifecycle gate is held from the liveness check until the reservations + /// are restored, so a removal cannot complete in between. + async fn restore( + &self, + wait: bool, + ) -> Result>, PlatformWalletError> { + if self.coordinator.network() != self.network { + return Err(PlatformWalletError::InvalidParameter(format!( + "the shielded coordinator is for {:?}, this wallet for {:?}", + self.coordinator.network(), + self.network + ))); + } + let _lifecycle = self.generation.payment_guard().await; + let current = { + let wm = self.wallet_manager.read().await; + wm.get_wallet_info(&self.wallet_id) + .is_some_and(|info| Arc::ptr_eq(&info.generation, &self.generation)) + }; + if !current { + return Err(PlatformWalletError::WalletNotFound(hex::encode( + self.wallet_id, + ))); + } + let dashpay_id = dashpay_contract()?.id(); + let (payments, journal_authoritative) = { + let mut store = if wait { + self.coordinator.store().write().await + } else { + match self.coordinator.store().try_write() { + Ok(store) => store, + Err(_) => return Ok(None), + } + }; + read_private_payment_journal(&mut *store, &self.persister, self.wallet_id, dashpay_id)? + }; + let restored = restore_journaled_private_payments( + &self.wallet_manager, + &self.generation, + &self.persister, + self.wallet_id, + &payments, + journal_authoritative, + ) + .await?; + // Opened only once a restoration succeeded: a restoration that failed + // closed over a deleted or replaced store fails the same way until a + // known-good store is back. One left closed here is opened by the + // next restoration, which no unresolved marker holds back now. + if !journal_authoritative { + let opened = if wait { + Some(self.coordinator.store().write().await) + } else { + self.coordinator.store().try_write().ok() + } + .map(|mut store| store.open_private_payment_journal(self.wallet_id)); + if let Some(Err(error)) = opened { + tracing::warn!( + wallet_id = %hex::encode(self.wallet_id), + %error, + "the private payment journal was not opened; the next restoration opens it" + ); + } + } + Ok(Some(restored)) + } +} + +/// Read `wallet_id`'s private payment journal for a restoration, as +/// (sender, recipient, account reference, entry commitment), with whether it +/// is authoritative (open: a restoration from it succeeded before). The +/// read-back notices are discharged first. While the journal is new, a spend +/// saved before it kept the wallet's payments is accounted for: one paying +/// for a private contact request with its request saved is added to the +/// journal, one with no request, or whose transition does not decode and +/// that no record accounts for, fails the read +/// ([`PlatformWalletError::ShieldedRecoveryCorrupted`]). +#[cfg(feature = "shielded")] +fn read_private_payment_journal( + store: &mut S, + persister: &WalletPersister, + wallet_id: WalletId, + dashpay_id: Identifier, +) -> Result<(Vec, bool), PlatformWalletError> +where + PlatformWalletError: From, +{ + discharge_read_back_private_payments(store, persister, wallet_id); + let journal_authoritative = store.private_payment_journal_open(wallet_id)?; + let mut records = store.private_payment_journal(wallet_id)?; + let journaled: std::collections::BTreeSet<[u8; 32]> = + records.iter().map(|record| record.commitment).collect(); + let accounted: std::collections::BTreeSet<[u8; 32]> = + records.iter().map(|record| record.activity_id).collect(); + let corrupted = + |account_index: u32, reason: &str| PlatformWalletError::ShieldedRecoveryCorrupted { + account_index: Some(account_index), + reason: reason.to_string(), + }; + // The private contact request payments saved before the journal kept + // them: type 29 spends to DashPay's store no record accounts for. + // While the journal is new only: once a restoration from it succeeded, + // every payment is journaled with its redrive record in one write, so a + // record no journal record accounts for paid for nothing, or was + // discharged. + let mut unjournaled = Vec::new(); + let saved_before_the_journal = if journal_authoritative { + Vec::new() + } else { + store.pending_redrives_for_wallet(wallet_id)? + }; + for (id, redrive) in saved_before_the_journal { + // Identity debits are never private document payments. + if redrive.nullifiers.is_empty() || accounted.contains(&redrive.activity_id) { + continue; + } + let transition = match StateTransition::deserialize_from_bytes_trusted(&redrive.st_bytes) { + Ok(StateTransition::PrivateDocumentCreateFromShieldedPool(transition)) => transition, + Ok(_) => continue, + Err(_) => { + return Err(corrupted( + id.account_index, + "a saved spend's transition does not decode, and no private payment journal \ + record shows it pays for no private contact request", + )) + } + }; + if transition.data_contract_id() == dashpay_id + && transition.document_type_name() == PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME + { + unjournaled.push((id, redrive)); + } + } + // Read only when needed, so a damaged association of a payment the + // journal keeps holds nothing up: for an unjournaled payment, and for a + // new journal, which accounts for nothing saved before it, damaged + // records retained without being loaded included. + let associations: BTreeMap<[u8; 32], PrivateContactPayment> = if journal_authoritative { + BTreeMap::new() + } else { + store + .private_contact_redrives_for_wallet(wallet_id)? + .into_iter() + .map(|(redrive, payment)| (redrive.activity_id, payment)) + .collect() + }; + let mut migrated = Vec::new(); + for (id, redrive) in unjournaled { + let Some(payment) = associations.get(&redrive.activity_id) else { + return Err(corrupted( + id.account_index, + "a private contact request payment saved before the journal kept them carries \ + no request, so its pair cannot be restored", + )); + }; + for commitment in associated_payment_commitments(&redrive, dashpay_id)? { + if !journaled.contains(&commitment) { + migrated.push(PrivatePaymentRecord { + account_index: id.account_index, + activity_id: redrive.activity_id, + anchor: redrive.anchor, + // Unknown for a payment saved before the journal: its + // expiry is never proven, so its pair stays held. + anchor_height: u64::MAX, + commitment, + payment: *payment, + nullifiers: redrive.nullifiers.clone(), + }); + } + } + } + if !migrated.is_empty() { + store.journal_private_payments(wallet_id, &migrated)?; + records.extend(migrated); + } + Ok(( + records + .into_iter() + .map(|record| { + ( + Identifier::from(record.payment.sender), + Identifier::from(record.payment.recipient), + record.payment.account_reference, + record.commitment, + ) + }) + .collect(), + journal_authoritative, + )) +} + +/// The entries of every private document create (type 29) among `redrives` +/// that appends to DashPay's `privateContactRequest` store, `dashpay_id` +/// being DashPay's contract id. A record whose transition does not decode +/// is skipped. +#[cfg(feature = "shielded")] +fn private_contact_request_redrive_entries<'a>( + redrives: impl IntoIterator, + dashpay_id: Identifier, +) -> Vec { + redrives + .into_iter() + .filter_map(|redrive| { + match StateTransition::deserialize_from_bytes_trusted(&redrive.st_bytes) { + Ok(StateTransition::PrivateDocumentCreateFromShieldedPool(transition)) + if transition.data_contract_id() == dashpay_id + && transition.document_type_name() + == PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME => + { + Some(transition.entries().to_vec()) + } + _ => None, + } + }) + .flatten() + .collect() +} + /// Verify a bech32m recipient's network class matches `network` before decoding. /// /// The address decoder is network-agnostic (`tdash` is shared by @@ -2381,6 +3186,267 @@ fn check_recipient_hrp( Ok(()) } +#[cfg(all(test, feature = "shielded"))] +mod private_contact_request_redrive_tests { + use super::*; + use dpp::serialization::PlatformSerializable; + use dpp::shielded::SerializedAction; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::methods::PrivateDocumentCreateFromShieldedPoolTransitionMethodsV0; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::PrivateDocumentCreateFromShieldedPoolTransition; + use dpp::version::PlatformVersion; + + /// A type 29 transition appending `entries` to the store of + /// `document_type_name` of `data_contract_id`, with placeholder actions. + fn private_document_create( + data_contract_id: Identifier, + document_type_name: &str, + entries: Vec, + ) -> StateTransition { + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + data_contract_id, + document_type_name.to_string(), + entries, + vec![action.clone(), action], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition") + } + + /// The unconfirmed spend that broadcast `st_bytes`. + fn redrive(st_bytes: Vec) -> PendingRedrive { + PendingRedrive { + activity_id: [1; 32], + anchor: [7; 32], + nullifiers: vec![[1; 32]], + st_bytes, + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + } + } + + /// Only the entries of type 29 transitions to DashPay's private contact + /// request store are read: not another contract's, not another document + /// type's, and not a record that does not decode. + #[test] + fn should_read_dashpays_private_contact_requests_from_unconfirmed_spends() { + let dashpay_id = dashpay_contract().expect("bundled DashPay").id(); + let entry = |byte: u8| BinaryData::new(vec![byte; 589]); + let serialized = |transition: StateTransition| { + transition + .serialize_to_bytes() + .expect("the transition serializes") + }; + let redrives = [ + redrive(serialized(private_document_create( + dashpay_id, + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + vec![entry(1), entry(2)], + ))), + redrive(serialized(private_document_create( + Identifier::new([4; 32]), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + vec![entry(3)], + ))), + redrive(serialized(private_document_create( + dashpay_id, + "privateNote", + vec![entry(4)], + ))), + redrive(vec![0xFF; 16]), + redrive(serialized(private_document_create( + dashpay_id, + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + vec![entry(5)], + ))), + ]; + assert_eq!( + private_contact_request_redrive_entries(&redrives, dashpay_id), + vec![entry(1), entry(2), entry(5)] + ); + assert_eq!( + private_contact_request_redrive_entries(&[] as &[PendingRedrive], dashpay_id), + Vec::::new() + ); + } +} + +#[cfg(all(test, feature = "shielded"))] +mod private_contact_request_payment_tests { + use super::*; + use crate::events::{EventHandler, PlatformEventHandler}; + use crate::wallet::persister::NoPlatformPersistence; + use crate::wallet::shielded::FileBackedShieldedStore; + use crate::PlatformWalletManager; + use grovedb_commitment_tree::ProvingKey; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + use std::time::Duration; + + const SEED: [u8; 64] = [0x42; 64]; + + struct NoopEventHandler; + impl EventHandler for NoopEventHandler {} + impl PlatformEventHandler for NoopEventHandler {} + + /// A prover the refused or unfunded payments here never reach. + struct UnreachableProver; + impl OrchardProver for UnreachableProver { + fn proving_key(&self) -> &ProvingKey { + panic!("nothing is proved") + } + } + + fn coordinator_at(dir: &std::path::Path) -> Arc { + let db_path = dir.join("tree.sqlite"); + let store = FileBackedShieldedStore::open_path(&db_path, 100).expect("open file store"); + Arc::new(NetworkShieldedCoordinator::new( + Arc::new(dash_sdk::Sdk::new_mock()), + dashcore::Network::Testnet, + db_path, + store, + )) + } + + async fn manager_with_wallet() -> ( + Arc>, + Arc, + ) { + let manager = Arc::new(PlatformWalletManager::new( + Arc::new(dash_sdk::Sdk::new_mock()), + Arc::new(NoPlatformPersistence), + Arc::new(NoopEventHandler), + )); + let wallet = add_wallet(&manager).await; + (manager, wallet) + } + + async fn add_wallet( + manager: &PlatformWalletManager, + ) -> Arc { + manager + .create_wallet_from_seed_bytes( + dashcore::Network::Testnet, + &SEED, + WalletAccountCreationOptions::Default, + Some(0), + ) + .await + .expect("wallet") + } + + /// Pay for a private contact request from `wallet`'s account 0. + async fn pay( + wallet: &PlatformWallet, + coordinator: &Arc, + ) -> Result { + let keyset = + OrchardKeySet::from_seed(&SEED, dashcore::Network::Testnet, 0).expect("keyset"); + wallet + .pay_private_contact_request( + coordinator, + keyset, + 0, + &Identifier::from([0xA1; 32]), + &Identifier::from([0xB1; 32]), + BinaryData::new(vec![1; 589]), + PaymentStarted::default(), + UnreachableProver, + ) + .await + } + + /// A handle to a wallet removed, or re-created under the same id, since + /// it was taken pays nothing for a private contact request: it is + /// refused at the liveness check, before the spend. + #[tokio::test] + async fn should_refuse_a_private_payment_from_a_wallet_no_longer_current() { + let dir = tempfile::tempdir().expect("temporary shielded store"); + let coordinator = coordinator_at(dir.path()); + let (manager, removed) = manager_with_wallet().await; + let wallet_id = removed.wallet_id(); + manager.remove_wallet(&wallet_id).await.expect("removed"); + let refused = pay(&removed, &coordinator).await; + assert!( + matches!(&refused, Err(PlatformWalletError::WalletRemovedBeforeSend(id)) + if *id == hex::encode(wallet_id)), + "{refused:?}" + ); + + let recreated = add_wallet(&manager).await; + assert_eq!(recreated.wallet_id(), wallet_id); + let refused = pay(&removed, &coordinator).await; + assert!( + matches!( + refused, + Err(PlatformWalletError::WalletRemovedBeforeSend(_)) + ), + "{refused:?}" + ); + let current = pay(&recreated, &coordinator).await; + assert!( + !matches!( + current, + Err(PlatformWalletError::WalletRemovedBeforeSend(_)) + ), + "the current wallet's payment goes on to the spend: {current:?}" + ); + } + + /// A removal waits for a private payment that entered its fenced section, + /// here held at the shielded store, and completes once it returns. + #[tokio::test] + async fn should_hold_a_wallets_removal_until_its_private_payment_returns() { + let dir = tempfile::tempdir().expect("temporary shielded store"); + let coordinator = coordinator_at(dir.path()); + let (manager, wallet) = manager_with_wallet().await; + let wallet_id = wallet.wallet_id(); + + let store = coordinator.store().write().await; + let payment = { + let wallet = Arc::clone(&wallet); + let coordinator = Arc::clone(&coordinator); + tokio::spawn(async move { pay(&wallet, &coordinator).await }) + }; + tokio::time::sleep(Duration::from_millis(100)).await; + let removal = { + let manager = Arc::clone(&manager); + tokio::spawn(async move { manager.remove_wallet(&wallet_id).await.map(|_| ()) }) + }; + tokio::time::sleep(Duration::from_millis(200)).await; + assert!(!payment.is_finished(), "the payment waits at the store"); + assert!( + !removal.is_finished(), + "the removal waits for the payment in flight" + ); + assert!(wallet.core().is_current_generation().await); + + drop(store); + let paid = payment.await.expect("payment task"); + assert!( + !matches!(paid, Err(PlatformWalletError::WalletRemovedBeforeSend(_))), + "the wallet was current when it was checked: {paid:?}" + ); + removal + .await + .expect("removal task") + .expect("the removal completes"); + assert!(!wallet.core().is_current_generation().await); + } +} + #[cfg(all(test, feature = "shielded"))] mod check_recipient_hrp_tests { use super::*; diff --git a/packages/rs-platform-wallet/src/wallet/shielded/balance_tests.rs b/packages/rs-platform-wallet/src/wallet/shielded/balance_tests.rs index 03adbe56175..b9700306b2e 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/balance_tests.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/balance_tests.rs @@ -365,6 +365,7 @@ async fn local_balance_restores_funds_and_durable_pending_reservations_offline() attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); diff --git a/packages/rs-platform-wallet/src/wallet/shielded/coordinator.rs b/packages/rs-platform-wallet/src/wallet/shielded/coordinator.rs index 884f44d8dda..2a53ce7ddbe 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/coordinator.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/coordinator.rs @@ -90,17 +90,22 @@ use super::balance::{ use super::file_store::FileBackedShieldedStore; use super::keys::AccountViewingKeys; use super::store::{ - IdentityDebitRecoveryRecord, IdentityDebitRecoveryStatus, ShieldedStore, StalePendingSpend, - SubwalletId, + IdentityDebitRecoveryRecord, IdentityDebitRecoveryStatus, PrivatePaymentRecord, ShieldedStore, + StalePendingSpend, SubwalletId, }; use super::CAUGHT_UP_COOLDOWN; use crate::error::PlatformWalletError; use crate::manager::shielded_sync::{ShieldedSyncPassSummary, WalletShieldedOutcome}; +use crate::wallet::identity::network::dashpay_contract; use crate::wallet::persister::WalletPersister; use crate::wallet::platform_wallet::WalletId; +use dash_sdk::platform::dashpay::PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME; +use dash_sdk::platform::private_document_scan::MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES; +use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::serialization::PlatformDeserializableTrusted; use dpp::state_transition::shield_from_identity_transition::accessors::ShieldFromIdentityTransitionAccessorsV0; use dpp::state_transition::StateTransition; +use futures::StreamExt; /// Network-scoped shielded coordinator. /// @@ -1329,6 +1334,31 @@ impl NetworkShieldedCoordinator { /// [`sync_notes_across`]: super::sync::sync_notes_across /// [`PlatformEventManager::on_shielded_sync_completed`]: /// crate::events::PlatformEventManager::on_shielded_sync_completed + /// Discharge the private payment journal records of the private contact + /// requests each registered wallet's private sync read back and recorded + /// durably (`WalletPersister::note_read_back_private_entries`). + async fn discharge_read_back_private_payments(&self) { + let noted: Vec<(WalletId, WalletPersister)> = self + .persisters + .read() + .await + .iter() + .filter(|(_, persister)| persister.has_read_back_private_entries()) + .map(|(wallet_id, persister)| (*wallet_id, persister.clone())) + .collect(); + if noted.is_empty() { + return; + } + let mut store = self.store.write().await; + for (wallet_id, persister) in ¬ed { + super::operations::discharge_read_back_private_payments( + &mut *store, + persister, + *wallet_id, + ); + } + } + pub async fn sync(&self, force: bool, cancel: &CancellationToken) -> ShieldedSyncPassSummary { // A stop that landed before any work: nothing to finalize, so // return the empty summary without touching Platform or the @@ -1340,6 +1370,10 @@ impl NetworkShieldedCoordinator { return ShieldedSyncPassSummary::default(); } + // Every pass, cooldown or not: the private requests a private sync read + // back discharge their journal records. + self.discharge_read_back_private_payments().await; + // Network-wide cooldown gate. Snapshot the remaining // window into a local before any await — `std::sync::Mutex` // is `!Send` across await points (clippy's @@ -1472,20 +1506,14 @@ impl NetworkShieldedCoordinator { // Runs before the balance read so freed notes are reflected in // this pass's balances. Skipped on a cancelled pass: the re-drive // broadcasts, and a skipped release simply waits for the next pass. - if let Some((snapshot, recorded)) = stranded_release.filter(|_| !cancelled) { - // Snapshot the per-wallet persisters BEFORE the loop and drop - // the read guard: `redrive_pending_spends` performs network - // broadcasts, and holding the persisters lock across those - // awaits would block wallet register/unregister for the - // duration of the round trips. - let subwallet_persisters: Vec<(SubwalletId, Option)> = { - let persisters = self.persisters.read().await; - subwallets - .iter() - .map(|(id, _)| (*id, persisters.get(&id.wallet_id).cloned())) - .collect() - }; - for (id, persister) in &subwallet_persisters { + if let Some(PreScanEvidence { + stale, + journal, + recorded, + recorded_height, + }) = stranded_release.filter(|_| !cancelled) + { + for (id, _) in &subwallets { // Network broadcasts: a stop that lands after the scan's // snapshot above stops the loop here, and the helper // checks the token before each request it starts. @@ -1495,8 +1523,6 @@ impl NetworkShieldedCoordinator { super::operations::redrive_pending_spends( &self.sdk, &self.store, - persister.as_ref(), - id.wallet_id, *id, &recorded, cancel, @@ -1504,7 +1530,14 @@ impl NetworkShieldedCoordinator { .await; } if !cancel.is_cancelled() { - self.release_stranded_spends(snapshot, &recorded).await; + let evidence = PrunedAnchorEvidence { + recorded: &recorded, + height: recorded_height, + scan_covered_height: notes.covered_height, + }; + self.release_stranded_spends(stale, evidence).await; + self.discharge_expired_private_payments(&journal, evidence) + .await; } } @@ -1654,30 +1687,37 @@ impl NetworkShieldedCoordinator { .unwrap_or(0) } - /// Pre-scan half of the stranded-reservation release: snapshot the - /// anchored reservations armed right now and fetch Platform's recorded - /// anchor set, or `None` when there is nothing to do. + /// Pre-scan half of the stranded-reservation release and of the private + /// payment expiry: snapshot the anchored reservations armed right now and + /// the wallets' private payment journal records, then fetch Platform's + /// recorded anchor set with the height of its proof, or `None` when there + /// is nothing to do. /// /// MUST run before the pass's note scan — the release's fund-safety /// argument ([`Self::release_stranded_spends`]) rests on both the - /// snapshot and the set predating the scan's spent-note reconcile. + /// snapshot and the set predating the scan's spent-note reconcile. The + /// journal is read before the set: a payment journaled after the fetch + /// may name a newer anchor than the set holds, so it is no candidate this + /// pass ([`Self::discharge_expired_private_payments`]). /// - /// Skips the network round-trip when no anchored reservation exists (the - /// common case), and treats a recorded-anchor fetch failure as - /// non-fatal — a sync must not fail because that query hiccupped; the - /// release simply waits for the next pass. + /// Skips the network round-trip when neither an anchored reservation nor + /// a journal record exists (the common case), and treats a + /// recorded-anchor fetch failure as non-fatal: a sync must not fail + /// because that query hiccupped, and the release simply waits for the + /// next pass. async fn prefetch_stranded_release( &self, subwallets: &[(SubwalletId, AccountViewingKeys)], - ) -> Option<(Vec<(SubwalletId, StalePendingSpend)>, HashSet<[u8; 32]>)> { - // Gather anchored reservations across every synced subwallet. The - // common case is none — then the network round-trip is skipped. - let stale: Vec<(SubwalletId, StalePendingSpend)> = { + ) -> Option { + // Gather anchored reservations across every synced subwallet, and the + // journal records of their wallets. The common case is none, and then + // the network round-trip is skipped. + let (stale, journal) = { let store = self.store.read().await; - let mut acc = Vec::new(); + let mut stale: Vec<(SubwalletId, StalePendingSpend)> = Vec::new(); for (id, _) in subwallets { match store.stale_pending_spends(*id) { - Ok(entries) => acc.extend(entries.into_iter().map(|entry| (*id, entry))), + Ok(entries) => stale.extend(entries.into_iter().map(|entry| (*id, entry))), Err(e) => tracing::warn!( wallet_id = %hex::encode(id.wallet_id), account = id.account_index, @@ -1686,16 +1726,25 @@ impl NetworkShieldedCoordinator { ), } } - acc + let journal = private_payment_journal_snapshot( + &*store, + subwallets.iter().map(|(id, _)| id.wallet_id), + ); + (stale, journal) }; - if stale.is_empty() { + if stale.is_empty() && journal.is_empty() { return None; } use dash_sdk::platform::fetch_current_no_parameters::FetchCurrent; - match dash_sdk::query_types::ShieldedAnchors::fetch_current(&self.sdk).await { - Ok(dash_sdk::query_types::ShieldedAnchors(anchors)) => { - Some((stale, anchors.into_iter().collect())) + match dash_sdk::query_types::ShieldedAnchors::fetch_current_with_metadata(&self.sdk).await { + Ok((dash_sdk::query_types::ShieldedAnchors(anchors), metadata)) => { + Some(PreScanEvidence { + stale, + journal, + recorded: anchors.into_iter().collect(), + recorded_height: metadata.height, + }) } Err(e) => { tracing::warn!( @@ -1716,27 +1765,30 @@ impl NetworkShieldedCoordinator { /// fact have landed), released only when the tx lands or the app /// restarts. A spend accepted but that never lands would otherwise /// strand its notes for the whole session. Here, after the pass's - /// spent-note reconcile, any reservation from the PRE-scan `snapshot` - /// whose anchor is absent from the PRE-scan `recorded` set is provably - /// dead: `validate_anchor_exists` accepts a spend only while its anchor - /// is retained (`shielded_anchor_retention_blocks = 1000`), so once the - /// anchor is pruned the transition can never execute. The notes are - /// freed and the linked activity row flipped to Failed so the UI shows - /// a clear, retryable failure instead of "Pending" forever. The - /// on-chain nullifier set stays the authoritative double-spend guard. + /// spent-note reconcile, a reservation from the PRE-scan `snapshot` is + /// released once `evidence` proves its anchor pruned + /// ([`PrunedAnchorEvidence::proves_pruned`]): `validate_anchor_exists` + /// accepts a spend only while its anchor is retained + /// (`shielded_anchor_retention_blocks = 1000`), so once the anchor is + /// pruned the transition can never execute. The notes are freed and the + /// linked activity row flipped to Failed so the UI shows a clear, + /// retryable failure instead of "Pending" forever. The on-chain + /// nullifier set stays the authoritative double-spend guard. /// - /// Fund-safe by construction, resting on two ordering facts: + /// Fund-safe by construction, resting on two facts: /// - /// 1. Both inputs predate the scan ([`Self::prefetch_stranded_release`]), - /// and the scan's spent-note reconcile ran to completion in between. - /// An anchor absent from the pre-scan set was pruned before the scan - /// began (a pruned root can never re-enter the set — the tree only - /// grows), so if that spend nevertheless executed, it executed at a - /// height the scan covered — the reconcile already cleared its - /// reservation, which fact 2 catches. + /// 1. The evidence is pinned to the anchor set's proved height H: the + /// reservation's anchor was in a set proved at or below H, so the set + /// at H lacks it only once it was pruned (an anchor absent but read + /// above H may be newer than the set), and the pass's note scan, + /// covering H, replayed every nullifier spent at or below H. So if + /// that spend executed, it did so at or below H, and the reconcile + /// already cleared its reservation, which fact 2 catches. Short of + /// that, as with a scan whose proofs are a block behind the set's, + /// nothing is released this pass. /// 2. Each release is a check-and-clear under a single store write /// guard: the reservation is cleared only while the nullifier is - /// still armed with the snapshot's exact anchor + activity. A + /// still armed with the snapshot's exact anchor, height + activity. A /// reservation resolved concurrently (the reconcile above, or the /// spend's own result-wait confirming mid-pass) — or cleared and /// re-armed by a retry with a fresh anchor — is skipped, and its @@ -1744,11 +1796,15 @@ impl NetworkShieldedCoordinator { /// /// A still-recorded (slow-but-landing) spend is never released, and an /// anchor-less reservation (reserved but not yet built) is never in the - /// snapshot (`stale_pending_spends` returns only anchored entries). + /// snapshot (`stale_pending_spends` returns only anchored entries). A + /// reservation whose anchor height is not known (one restored from a + /// record saved before heights were kept) gets H once the set at H + /// holds its anchor ([`ShieldedStore::note_anchor_recorded`]), and is + /// released only on a later set that lacks it. async fn release_stranded_spends( &self, snapshot: Vec<(SubwalletId, StalePendingSpend)>, - recorded: &HashSet<[u8; 32]>, + evidence: PrunedAnchorEvidence<'_>, ) { // Persister map is only written on wallet register/unregister — // one read acquisition covers the whole loop. @@ -1756,10 +1812,18 @@ impl NetworkShieldedCoordinator { // Every note of a multi-note spend carries the same activity id: // flip each linked row once, not once per nullifier. let mut flipped: HashSet<[u8; 32]> = HashSet::new(); - for (id, (nullifier, anchor, activity_id)) in snapshot { - // Fund-safety invariant: release ONLY when the anchor is absent - // from the recorded set. A still-recorded anchor may yet land. - if recorded.contains(&anchor) { + // Reservations whose anchor height is not known, and whose anchor the + // set at H holds: given H, a later set lacking it shows it pruned. + let mut seen_recorded: std::collections::BTreeSet<(SubwalletId, [u8; 32])> = + std::collections::BTreeSet::new(); + for (id, (nullifier, anchor, activity_id, anchor_height)) in snapshot { + if anchor_height.is_none() && evidence.recorded.contains(&anchor) { + seen_recorded.insert((id, anchor)); + } + // Fund-safety invariant: release ONLY on proof the anchor was + // pruned with the scan covering it. A still-recorded anchor may + // yet land; an absent one read above H may be newer than the set. + if !evidence.proves_pruned(&anchor, anchor_height) { continue; } // Check-and-clear under ONE write acquisition: the snapshot @@ -1771,9 +1835,12 @@ impl NetworkShieldedCoordinator { { let mut store = self.store.write().await; let still_same = match store.stale_pending_spends(id) { - Ok(current) => current - .iter() - .any(|(n, a, act)| *n == nullifier && *a == anchor && *act == activity_id), + Ok(current) => current.iter().any(|(n, a, act, height)| { + *n == nullifier + && *a == anchor + && *act == activity_id + && *height == anchor_height + }), Err(e) => { tracing::warn!( wallet_id = %hex::encode(id.wallet_id), @@ -1839,6 +1906,175 @@ impl NetworkShieldedCoordinator { .await; } } + if !seen_recorded.is_empty() { + let mut store = self.store.write().await; + for (id, anchor) in seen_recorded { + if let Err(e) = store.note_anchor_recorded(id, &anchor, evidence.height) { + tracing::warn!( + wallet_id = %hex::encode(id.wallet_id), + account = id.account_index, + error = %e, + "shielded reservation release: the anchor's recorded height was not \ + kept; its reservations stay until it is" + ); + } + } + } + } + + /// Discharge the private payment journal records of payments proven + /// expired, durably, and note them on their wallet's persister so the + /// wallet releases the pairs they hold + /// (`WalletPersister::note_discharged_private_entries`). + /// + /// The candidates are `journal`, the records the pass read before it + /// fetched the anchor set `evidence` holds, proved at H. A payment is + /// proven never to execute when: + /// + /// 1. its record still reads in the journal exactly as `journal` has it: + /// a payment journaled since, or armed again, may name an anchor newer + /// than the set; + /// 2. `evidence` proves its anchor pruned + /// ([`PrunedAnchorEvidence::proves_pruned`], the rule the note release + /// judges by): read from a set proved at or below H + /// (`anchor_height`), absent from the set at H, and the pass's note + /// scan covering H. A pruned root never re-enters the set, so the + /// transition cannot execute above H + /// (`shielded_anchor_retention_blocks` 1000, pruned every + /// `shielded_anchor_pruning_interval` 100 blocks); + /// 3. its entry is absent from DashPay's private contact request store, + /// read raw through its end with every page proved at or above H + /// ([`Self::private_contact_requests_held`]): an entry starts with its + /// commitment, so none needs opening. The transition appends its entry + /// in the block that spends its inputs, so an entry absent at or above + /// H, the transition unable to execute above H, shows it never + /// executed. Whether its inputs are spent does not matter: a competing + /// spend that consumed them does not keep its pair held. + /// + /// Anything short of that discharges nothing this pass, and the next pass + /// retries: a scan or a store read below H, a read that fails, a record + /// changed since. A relay refusal or a missing redrive record is never + /// taken as proof. Point 1 is checked again under the store write guard + /// that discharges the record. + pub(crate) async fn discharge_expired_private_payments( + &self, + journal: &[(WalletId, PrivatePaymentRecord)], + evidence: PrunedAnchorEvidence<'_>, + ) { + let recorded_height = evidence.height; + let pruned: Vec<&(WalletId, PrivatePaymentRecord)> = journal + .iter() + .filter(|(_, record)| { + evidence.proves_pruned(&record.anchor, Some(record.anchor_height)) + }) + .collect(); + if pruned.is_empty() { + return; + } + let candidates: Vec<(WalletId, PrivatePaymentRecord)> = { + let store = self.store.read().await; + pruned + .into_iter() + .filter(|(wallet_id, record)| unchanged(&*store, *wallet_id, record)) + .cloned() + .collect() + }; + if candidates.is_empty() { + return; + } + let commitments: HashSet<[u8; 32]> = candidates + .iter() + .map(|(_, record)| record.commitment) + .collect(); + let held = match self + .private_contact_requests_held(&commitments, recorded_height) + .await + { + Ok(held) => held, + Err(error) => { + tracing::warn!( + %error, + "private payment expiry: the private contact request store was not read at \ + the anchor set's height; retried next pass" + ); + return; + } + }; + let persisters = self.persisters.read().await; + let mut store = self.store.write().await; + for (wallet_id, record) in candidates { + if held.contains(&record.commitment) || !unchanged(&*store, wallet_id, &record) { + continue; + } + match store.discharge_private_payment(wallet_id, &record.commitment) { + Ok(_) => { + tracing::info!( + wallet_id = %hex::encode(wallet_id), + commitment = %hex::encode(record.commitment), + "a private payment's anchor was pruned and its entry is absent from the \ + store: it never executed and never can, its pair is freed" + ); + if let Some(persister) = persisters.get(&wallet_id) { + persister.note_discharged_private_entries([record.commitment]); + } + } + Err(error) => tracing::warn!( + wallet_id = %hex::encode(wallet_id), + %error, + "an expired private payment's journal record was not discharged; retried \ + next pass" + ), + } + } + } + + /// Which of `candidates` DashPay's private contact request store holds, + /// read now from its start through the end its first page shows, raw: an + /// entry starts with its commitment. Only the matches are kept, though + /// every entry is read to show the others absent. Every page must be + /// proved at or above `floor`, the first page's end with it: a page below + /// fails the read, as the store it shows may predate the floor. + pub(crate) async fn private_contact_requests_held( + &self, + candidates: &HashSet<[u8; 32]>, + floor: u64, + ) -> Result, PlatformWalletError> { + let contract_id = dashpay_contract()?.id(); + let mut pages = std::pin::pin!(self.sdk.private_document_pages_with_heights( + contract_id, + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES, + )); + let mut held = HashSet::new(); + let mut read = false; + while let Some(page) = pages.next().await { + let (page, height) = page?; + if height < floor { + return Err(PlatformWalletError::ShieldedSyncFailed(format!( + "the private contact request store's page from position {} is proved at \ + height {height}, below {floor}", + page.start_position + ))); + } + read = true; + for entry in page.entries.entries { + if let Some(commitment) = entry + .data + .get(..32) + .and_then(|commitment| <[u8; 32]>::try_from(commitment).ok()) + .filter(|commitment| candidates.contains(commitment)) + { + held.insert(commitment); + } + } + } + if !read { + return Err(PlatformWalletError::ShieldedSyncFailed( + "the private contact request store returned no page".to_string(), + )); + } + Ok(held) } /// Derive best-effort activity entries from each subwallet's @@ -2239,6 +2475,94 @@ fn is_own_orchard_recipient(views: &AccountViewingKeys, recipient: &[u8]) -> boo .is_some() } +/// The evidence a sync pass has that an anchor was pruned: Platform's recorded +/// anchor set `recorded`, proved at `height` (H) and read before the pass's +/// note scan, and the height that scan covers (`scan_covered_height`, +/// `MultiSyncNotesResult::covered_height`). The one rule the note release +/// ([`NetworkShieldedCoordinator::release_stranded_spends`]) and the private +/// payment expiry +/// ([`NetworkShieldedCoordinator::discharge_expired_private_payments`]) judge +/// an anchor by. +#[derive(Clone, Copy)] +pub(crate) struct PrunedAnchorEvidence<'a> { + pub(crate) recorded: &'a HashSet<[u8; 32]>, + pub(crate) height: u64, + pub(crate) scan_covered_height: Option, +} + +impl PrunedAnchorEvidence<'_> { + /// Whether a spend built on `anchor`, which was in Platform's recorded + /// set at `anchor_height`, is proven unable to execute above H, with all + /// it could have done at or below H seen by the note scan: + /// + /// 1. `anchor_height` is known and at or below H, so the set at H holds + /// the anchor unless it was pruned, and the set lacks it: a pruned + /// root never re-enters the set, so the spend cannot execute above H. + /// An anchor read above H, or at an unknown height, may simply be + /// newer than the set. + /// 2. the scan covers H: a spend that executed at or below H spent its + /// nullifiers where the scan replayed them. + pub(crate) fn proves_pruned(&self, anchor: &[u8; 32], anchor_height: Option) -> bool { + anchor_height.is_some_and(|read| read <= self.height) + && !self.recorded.contains(anchor) + && self + .scan_covered_height + .is_some_and(|covered| covered >= self.height) + } +} + +/// What a sync pass reads before its note scan, for the steps after it that +/// may act only on evidence from before the scan +/// ([`NetworkShieldedCoordinator::prefetch_stranded_release`]). +struct PreScanEvidence { + /// The anchored reservations, for + /// [`NetworkShieldedCoordinator::release_stranded_spends`]. + stale: Vec<(SubwalletId, StalePendingSpend)>, + /// The private payment journal records of the synced wallets, read before + /// the anchor set was fetched, for + /// [`NetworkShieldedCoordinator::discharge_expired_private_payments`]. + journal: Vec<(WalletId, PrivatePaymentRecord)>, + /// Platform's recorded anchor set. + recorded: HashSet<[u8; 32]>, + /// The height of the proof the anchor set was read from. + recorded_height: u64, +} + +/// The private payment journal records of `wallet_ids`, each wallet once, in +/// order. A wallet whose journal does not read has none here: nothing of it is +/// a candidate for expiry. +pub(crate) fn private_payment_journal_snapshot( + store: &S, + wallet_ids: impl IntoIterator, +) -> Vec<(WalletId, PrivatePaymentRecord)> { + let wallets: std::collections::BTreeSet = wallet_ids.into_iter().collect(); + let mut journal = Vec::new(); + for wallet_id in wallets { + match store.private_payment_journal(wallet_id) { + Ok(records) => journal.extend(records.into_iter().map(|record| (wallet_id, record))), + Err(error) => tracing::warn!( + wallet_id = %hex::encode(wallet_id), + %error, + "private payment expiry: the journal does not read; no payment of the wallet \ + is a candidate this pass" + ), + } + } + journal +} + +/// Whether `wallet_id`'s journal still holds `record` exactly as a pass's +/// snapshot read it. A store read that fails says no. +fn unchanged( + store: &S, + wallet_id: WalletId, + record: &PrivatePaymentRecord, +) -> bool { + store + .private_payment_journal(wallet_id) + .is_ok_and(|records| records.contains(record)) +} + #[cfg(test)] mod tests { use super::*; @@ -2350,6 +2674,7 @@ mod tests { attempts: 0, identity_nonce_finalized: true, identity_user_abandoned: false, + anchor_height: None, }, cmxs, ) @@ -2923,6 +3248,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; coordinator .store() @@ -3299,6 +3625,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; coordinator .store() @@ -3412,6 +3739,7 @@ mod tests { attempts: 0, identity_nonce_finalized: true, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -3758,24 +4086,31 @@ mod tests { let mut store = coordinator.store.write().await; store.mark_pending(id, &pruned_nf).expect("mark pruned"); store - .set_pending_spend(id, &pruned_nf, pruned_anchor, pruned_act) + .set_pending_spend(id, &pruned_nf, pruned_anchor, pruned_act, Some(10)) .expect("arm pruned"); store.mark_pending(id, &live_nf).expect("mark live"); store - .set_pending_spend(id, &live_nf, live_anchor, live_act) + .set_pending_spend(id, &live_nf, live_anchor, live_act, Some(10)) .expect("arm live"); } // Snapshot as the pre-scan prefetch would have produced it; the // recorded set holds only the live anchor (the other was "pruned"). let snapshot = vec![ - (id, (pruned_nf, pruned_anchor, Some(pruned_act))), - (id, (live_nf, live_anchor, Some(live_act))), + (id, (pruned_nf, pruned_anchor, Some(pruned_act), Some(10))), + (id, (live_nf, live_anchor, Some(live_act), Some(10))), ]; let recorded: HashSet<[u8; 32]> = [live_anchor].into_iter().collect(); coordinator - .release_stranded_spends(snapshot, &recorded) + .release_stranded_spends( + snapshot, + PrunedAnchorEvidence { + recorded: &recorded, + height: 10, + scan_covered_height: Some(10), + }, + ) .await; let remaining = coordinator @@ -3797,6 +4132,279 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + /// The whole sync pass, its note scan proved a block below the anchor set + /// (within the SDK's one-block tolerance): a spend built on an anchor the + /// set lacks may have executed in that block, which the scan has not + /// shown, so its note stays reserved, its saved record stays and its + /// activity stays Pending. Once the scan covers the set's height, the + /// pass releases it. An anchor read above the set's height, absent from + /// it, may be newer than the set and is never released by it. + #[tokio::test] + async fn should_release_a_stranded_spend_only_once_the_scan_covers_the_anchor_set() { + use super::super::activity::{ + ShieldedActivityEntry, ShieldedActivityKind, ShieldedDirection, + }; + use crate::wallet::shielded::ShieldedNote; + use drive_proof_verifier::types::{ + NoParamQuery, ShieldedAnchors, ShieldedEncryptedNotes, ShieldedEncryptedNotesQuery, + }; + + const SET_HEIGHT: u64 = 50; + let mut sdk = dash_sdk::Sdk::new_mock(); + let count = 2048 + * u32::from( + sdk.version() + .drive_abci + .query + .shielded_queries + .max_query_chunks, + ); + // The scan's window of 16 chunk fetches, all of an empty pool. + for chunk in 0..16 { + sdk.mock() + .expect_fetch( + ShieldedEncryptedNotesQuery { + start_index: u64::from(count) * chunk, + count, + }, + Some(ShieldedEncryptedNotes { + notes: Vec::new(), + total_count: 0, + }), + ) + .await + .expect("an empty chunk"); + } + sdk.mock() + .expect_fetch(NoParamQuery {}, Some(ShieldedAnchors(vec![[0xEE; 32]]))) + .await + .expect("the anchor set"); + sdk.mock() + .set_response_height_of::(SET_HEIGHT) + .set_response_height_of::(SET_HEIGHT - 1); + + let dir = temp_dir("release_scan_below_set"); + std::fs::create_dir_all(&dir).expect("create temp dir"); + let db_path = dir.join("tree.sqlite"); + let coordinator = NetworkShieldedCoordinator::new( + Arc::new(sdk.clone()), + dashcore::Network::Testnet, + db_path.clone(), + FileBackedShieldedStore::open_path(&db_path, 100).expect("open file store"), + ); + let wallet_id: WalletId = [0x11; 32]; + let views = OrchardKeySet::from_seed(&[0x42u8; 64], dashcore::Network::Testnet, 0) + .expect("derive viewing keys") + .viewing_keys(); + coordinator + .register_wallet( + wallet_id, + BTreeMap::from([(0u32, views)]), + WalletPersister::new(wallet_id, Arc::new(NoPlatformPersistence)), + ) + .await + .expect("register wallet"); + let id = SubwalletId::new(wallet_id, 0); + // Two broadcast spends whose anchors the set lacks: one read below + // the set's height, one above it. + let spends = [ + ([0x21; 32], [0x22; 32], [0x23; 32], SET_HEIGHT - 5), + ([0x31; 32], [0x32; 32], [0x33; 32], SET_HEIGHT + 1), + ]; + { + let mut store = coordinator.store().write().await; + for (index, (nullifier, anchor, activity, anchor_height)) in spends.iter().enumerate() { + let note = ShieldedNote { + position: index as u64, + cmx: [0x40 + index as u8; 32], + nullifier: *nullifier, + block_height: 1, + is_spent: false, + value: 1_000, + note_data: vec![0u8; 115], + }; + store.save_note(id, ¬e).expect("note"); + store.mark_pending(id, nullifier).expect("reserved"); + store + .set_pending_spend(id, nullifier, *anchor, *activity, Some(*anchor_height)) + .expect("anchored"); + store + .arm_redrive( + id, + PendingRedrive { + activity_id: *activity, + anchor: *anchor, + nullifiers: vec![*nullifier], + st_bytes: vec![0xDE, 0xAD], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: Some(*anchor_height), + }, + ) + .expect("record"); + store + .save_activity( + id, + &ShieldedActivityEntry { + id: *activity, + kind: ShieldedActivityKind::Unshield, + direction: ShieldedDirection::Out, + amount: 900, + fee: None, + counterparty: None, + memo: None, + block_height: None, + status: ShieldedActivityStatus::Pending, + created_at_ms: 1, + min_note_position: None, + note_cmxs: vec![], + spent_nullifiers: vec![*nullifier], + }, + ) + .expect("activity"); + } + } + let state = |store: &FileBackedShieldedStore, nullifier: [u8; 32], activity: [u8; 32]| { + let reserved = !store + .get_unspent_notes(id) + .expect("notes") + .iter() + .any(|note| note.nullifier == nullifier); + let saved = store + .pending_redrives(id) + .expect("records") + .iter() + .any(|record| record.activity_id == activity); + let status = store + .get_activity_by_entry_id(id, &activity) + .expect("activity") + .map(|entry| entry.status); + (reserved, saved, status) + }; + let held = (true, true, Some(ShieldedActivityStatus::Pending)); + + coordinator.sync(true, &CancellationToken::new()).await; + { + let store = coordinator.store().read().await; + for (nullifier, _, activity, _) in &spends { + assert_eq!( + state(&store, *nullifier, *activity), + held, + "a scan a block below the set releases nothing" + ); + } + } + + sdk.mock() + .set_response_height_of::(SET_HEIGHT); + coordinator.sync(true, &CancellationToken::new()).await; + let store = coordinator.store().read().await; + assert_eq!( + state(&store, spends[0].0, spends[0].2), + (false, false, Some(ShieldedActivityStatus::Failed)), + "proven pruned once the scan covers the set" + ); + assert_eq!( + state(&store, spends[1].0, spends[1].2), + held, + "an anchor read above the set may be newer than it" + ); + drop(store); + let _ = std::fs::remove_dir_all(&dir); + } + + /// A reservation restored from a record saved before anchor heights were + /// kept has no known height: its anchor absent from a set may be newer + /// than it, so it is not released. A set holding the anchor gives it that + /// set's height, durably, and a later set lacking it, with the scan + /// covering that set, releases it. + #[tokio::test] + async fn should_release_a_heightless_reservation_only_after_a_set_held_its_anchor() { + let dir = temp_dir("release_heightless"); + let coordinator = coordinator_with_one_wallet(&dir).await; + let id = SubwalletId::new([0x11; 32], 0); + let nullifier = [0x51; 32]; + let anchor = [0x52; 32]; + let activity = [0x53; 32]; + { + let mut store = coordinator.store.write().await; + store.mark_pending(id, &nullifier).expect("reserved"); + store + .set_pending_spend(id, &nullifier, anchor, activity, None) + .expect("anchored, no height"); + store + .arm_redrive( + id, + PendingRedrive { + activity_id: activity, + anchor, + nullifiers: vec![nullifier], + st_bytes: vec![0xDE, 0xAD], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + ) + .expect("record"); + } + let stale = |coordinator: &NetworkShieldedCoordinator| { + let store = coordinator.store.clone(); + async move { store.read().await.stale_pending_spends(id).expect("stale") } + }; + let release = |recorded: HashSet<[u8; 32]>, height: u64| { + let snapshot_store = coordinator.store.clone(); + let coordinator = &coordinator; + async move { + let snapshot: Vec<(SubwalletId, StalePendingSpend)> = snapshot_store + .read() + .await + .stale_pending_spends(id) + .expect("stale") + .into_iter() + .map(|entry| (id, entry)) + .collect(); + coordinator + .release_stranded_spends( + snapshot, + PrunedAnchorEvidence { + recorded: &recorded, + height, + scan_covered_height: Some(height), + }, + ) + .await; + } + }; + + // Absent from the first set seen: perhaps newer than it. + release(HashSet::new(), 10).await; + assert_eq!( + stale(&coordinator).await, + vec![(nullifier, anchor, Some(activity), None)], + "not proven pruned" + ); + // A set holding it gives it the set's height. + release(HashSet::from([anchor]), 20).await; + assert_eq!( + stale(&coordinator).await, + vec![(nullifier, anchor, Some(activity), Some(20))], + ); + let reopened = + FileBackedShieldedStore::open_path(dir.join("tree.sqlite"), 100).expect("reopened"); + assert_eq!( + reopened.pending_redrives(id).expect("records")[0].anchor_height, + Some(20), + "kept durably" + ); + drop(reopened); + // A later set lacking it releases it. + release(HashSet::new(), 30).await; + assert!(stale(&coordinator).await.is_empty(), "proven pruned"); + let _ = std::fs::remove_dir_all(&dir); + } + /// A token cancelled before the pass starts: no network call (the /// mock SDK would otherwise fail every wallet), no store change and /// no caught-up cooldown stamp (dashpay/platform#5200). diff --git a/packages/rs-platform-wallet/src/wallet/shielded/file_store.rs b/packages/rs-platform-wallet/src/wallet/shielded/file_store.rs index 0970e75d48a..799d152febc 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/file_store.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/file_store.rs @@ -11,7 +11,7 @@ //! tree (or, when the host persister is wired up, restored from //! SwiftData before sync runs). -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use std::error::Error as StdError; use std::fmt; use std::path::{Path, PathBuf}; @@ -23,12 +23,20 @@ use grovedb_commitment_tree::{ClientPersistentCommitmentTree, Position, Retentio use super::balance::{ShieldedBalanceSource, ShieldedLocalAccountBalance}; use super::store::{ - PendingRedrive, ShieldedNote, ShieldedOutgoingNote, ShieldedStore, StalePendingSpend, - SubwalletId, SubwalletState, + PendingRedrive, PrivateContactPayment, PrivatePaymentRecord, ShieldedNote, + ShieldedOutgoingNote, ShieldedStore, StalePendingSpend, SubwalletId, SubwalletState, }; use crate::error::PlatformWalletError; use crate::wallet::platform_wallet::WalletId; +/// The note-spend kind of a private document create from the shielded pool +/// (type 29), the transition a private contact request payment is. +const PRIVATE_DOCUMENT_CREATE_KIND: &str = "private document create from shielded pool"; + +/// A private payment journal row: the entry's commitment, which keys it, and +/// the record's bytes ([`PrivatePaymentRecord::to_bytes`]). +type JournalRow = ([u8; 32], Vec); + /// Error type for [`FileBackedShieldedStore`]. #[derive(Debug)] pub enum FileShieldedStoreError { @@ -102,16 +110,242 @@ pub struct FileBackedShieldedStore { /// Per-subwallet notes + sync state, keyed by `(wallet_id, /// account_index)`. Lazily populated on first use of an id. subwallets: BTreeMap, - /// Second connection on the same SQLite file, owning the - /// `shielded_pending_spends` table (armed [`PendingRedrive`] - /// records). Separate from `tree` because the commitment-tree - /// wrapper takes its `Connection` by value; WAL mode makes the - /// two-connection setup safe. `Mutex` for the same `Sync`-shim - /// reason as `tree`. + /// Second connection on the same SQLite file, owning the recovery + /// tables: `shielded_pending_spends` (armed [`PendingRedrive`] records) + /// and the private payment journal. Separate from `tree` because the + /// commitment-tree wrapper takes its `Connection` by value; WAL mode + /// makes the two-connection setup safe. It commits at + /// `synchronous=FULL` ([`Self::open_recovery_connection`]): a recovery + /// record saved before a broadcast must survive a power loss, unlike the + /// rebuildable tree. `Mutex` for the same `Sync`-shim reason as `tree`. pending_conn: Mutex, + /// The private contact request each armed record pays for, by subwallet + /// and activity id, as the `private_contact_payment` column keeps it. + /// Read joined with the live records, so an entry whose record was + /// dropped is passed over. A record with no association (SQL NULL, as + /// one saved before associations were kept) has no entry. + private_contact_payments: BTreeMap<(SubwalletId, [u8; 32]), SavedPrivateContactPayment>, + /// The wallets of damaged records retained but not loaded that carry a + /// private contact request association, `None` for one whose wallet does + /// not read either: the payment they stand for cannot be restored, so + /// restoring a covered wallet's reservations fails + /// ([`ShieldedStore::private_contact_redrives_for_wallet`]). + unloaded_private_contact_payments: BTreeSet>, + /// The private payment journal's records, by wallet and commitment, as + /// the `shielded_private_payment_journal` table keeps them. + private_payment_journal: BTreeMap<(WalletId, [u8; 32]), PrivatePaymentRecord>, + /// The wallets whose private payment journal is open, as the + /// `shielded_private_payment_journal_wallets` table keeps them. + private_payment_journals: BTreeSet, + /// The wallets of journal records that do not read back, `None` for one + /// whose wallet does not read either. The row stays, and reading a + /// covered wallet's journal fails, so its holds stay. + unreadable_private_payment_journal: BTreeSet>, +} + +/// The private contact request association a saved record carries. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum SavedPrivateContactPayment { + /// The association, as it was saved. + Readable(PrivateContactPayment), + /// A non-null association that does not decode, or a column that does + /// not read: restoring the record's wallet fails, so its holds stay. + Unreadable, } impl FileBackedShieldedStore { + /// Drop the private contact request associations of `id` whose record is + /// gone, as its redrives are removed. The listing joins with the live + /// records, so a left one restores nothing, but would stay in memory + /// until the store is reopened. + fn evict_private_contact_payments(&mut self, id: SubwalletId) { + let live = self.subwallets.get(&id); + self.private_contact_payments + .retain(|(owner, activity_id), _| { + *owner != id || live.is_some_and(|sw| sw.redrives.contains_key(activity_id)) + }); + } + + /// Persist `redrive`, with the private contact request it pays for when + /// it does, and the journal `records` of the entries it appends, in one + /// transaction committed at `synchronous=FULL`. + fn arm_redrive_paying_for( + &mut self, + id: SubwalletId, + redrive: PendingRedrive, + payment: Option, + records: &[PrivatePaymentRecord], + ) -> Result<(), FileShieldedStoreError> { + let journal_rows = Self::journal_rows(records)?; + { + let mut conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + let tx = conn + .transaction() + .map_err(|e| FileShieldedStoreError::Storage(format!("begin redrive: {e}")))?; + let nullifier_blob: Vec = redrive.nullifiers.iter().flatten().copied().collect(); + tx.execute( + "INSERT OR REPLACE INTO shielded_pending_spends \ + (wallet_id, account_index, activity_id, anchor, nullifiers, st_bytes, attempts, identity_nonce_finalized, identity_user_abandoned, private_contact_payment, anchor_height) \ + VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)", + rusqlite::params![ + id.wallet_id.as_slice(), + id.account_index, + redrive.activity_id.as_slice(), + redrive.anchor.as_slice(), + nullifier_blob, + redrive.st_bytes, + redrive.attempts, + redrive.identity_nonce_finalized, + redrive.identity_user_abandoned, + payment.map(|payment| payment.to_bytes().to_vec()), + // A height past SQLite's integer range is kept unknown, + // which only holds the notes longer. + redrive + .anchor_height + .and_then(|height| i64::try_from(height).ok()), + ], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("persist redrive: {e}")))?; + Self::insert_journal_rows(&tx, id.wallet_id, &journal_rows)?; + tx.commit() + .map_err(|e| FileShieldedStoreError::Storage(format!("commit redrive: {e}")))?; + } + if !records.is_empty() { + self.private_payment_journals.insert(id.wallet_id); + } + for record in records { + self.private_payment_journal + .insert((id.wallet_id, record.commitment), record.clone()); + } + match payment { + Some(payment) => { + self.private_contact_payments.insert( + (id, redrive.activity_id), + SavedPrivateContactPayment::Readable(payment), + ); + } + None => { + self.private_contact_payments + .remove(&(id, redrive.activity_id)); + } + } + self.subwallets.entry(id).or_default().arm_redrive(redrive); + Ok(()) + } + + /// The rows `records` are kept as: commitment and record bytes. Fails + /// for a record [`PrivatePaymentRecord::to_bytes`] refuses. + fn journal_rows( + records: &[PrivatePaymentRecord], + ) -> Result, FileShieldedStoreError> { + records + .iter() + .map(|record| { + record + .to_bytes() + .map(|bytes| (record.commitment, bytes)) + .ok_or_else(|| { + FileShieldedStoreError::Storage( + "a private payment's journal record spends no note, or too many" + .to_string(), + ) + }) + }) + .collect() + } + + /// Insert `rows` into `wallet_id`'s journal within `tx`, opening the + /// journal too. + fn insert_journal_rows( + tx: &rusqlite::Transaction<'_>, + wallet_id: WalletId, + rows: &[JournalRow], + ) -> Result<(), FileShieldedStoreError> { + if rows.is_empty() { + return Ok(()); + } + tx.execute( + "INSERT OR IGNORE INTO shielded_private_payment_journal_wallets (wallet_id) \ + VALUES (?1)", + rusqlite::params![wallet_id.as_slice()], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("open payment journal: {e}")))?; + for (commitment, bytes) in rows { + tx.execute( + "INSERT OR REPLACE INTO shielded_private_payment_journal \ + (wallet_id, commitment, record) VALUES (?1, ?2, ?3)", + rusqlite::params![wallet_id.as_slice(), commitment.as_slice(), bytes], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("journal payment: {e}")))?; + } + Ok(()) + } + + /// Load the private payment journal into memory. A row that does not + /// read back stays, and flags its wallet + /// (`unreadable_private_payment_journal`); the open does not fail. + fn load_private_payment_journal(&mut self) -> Result<(), FileShieldedStoreError> { + let conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + let mut journals = conn + .prepare("SELECT wallet_id FROM shielded_private_payment_journal_wallets") + .map_err(|e| FileShieldedStoreError::Storage(format!("prepare journals: {e}")))?; + let wallets = journals + .query_map([], |row| row.get::<_, Vec>(0)) + .map_err(|e| FileShieldedStoreError::Storage(format!("query journals: {e}")))?; + for wallet in wallets { + // A journal whose wallet does not read is not open for any wallet. + if let Some(wallet_id) = wallet + .ok() + .and_then(|bytes| <[u8; 32]>::try_from(bytes.as_slice()).ok()) + { + self.private_payment_journals.insert(wallet_id); + } + } + drop(journals); + let mut statement = conn + .prepare("SELECT wallet_id, commitment, record FROM shielded_private_payment_journal") + .map_err(|e| FileShieldedStoreError::Storage(format!("prepare journal: {e}")))?; + let rows = statement + .query_map([], |row| { + Ok(( + row.get::<_, Vec>(0).ok(), + row.get::<_, Vec>(1).ok(), + row.get::<_, Vec>(2).ok(), + )) + }) + .map_err(|e| FileShieldedStoreError::Storage(format!("query journal: {e}")))?; + for row in rows { + let (wallet_id, commitment, record) = + row.map_err(|e| FileShieldedStoreError::Storage(format!("read journal: {e}")))?; + let wallet_id = wallet_id.and_then(|bytes| <[u8; 32]>::try_from(bytes.as_slice()).ok()); + let read = wallet_id.and_then(|wallet_id| { + let commitment = <[u8; 32]>::try_from(commitment?.as_slice()).ok()?; + let record = PrivatePaymentRecord::from_bytes(&record?)?; + (record.commitment == commitment).then_some((wallet_id, record)) + }); + match read { + Some((wallet_id, record)) => { + self.private_payment_journal + .insert((wallet_id, record.commitment), record); + } + None => { + tracing::warn!( + "retaining a private payment journal record that does not read back; \ + its wallet's private contact request holds stay" + ); + self.unreadable_private_payment_journal.insert(wallet_id); + } + } + } + Ok(()) + } + /// Called under the coordinator's store read guard so amount and provenance /// cannot straddle a sync or a pending-spend reservation. pub(super) fn local_account_balance( @@ -174,7 +408,7 @@ impl FileBackedShieldedStore { let conn = Self::open_tuned_connection(&path)?; let tree = ClientPersistentCommitmentTree::open(conn, max_checkpoints) .map_err(|e| FileShieldedStoreError::Storage(format!("open commitment tree: {e}")))?; - let pending_conn = Self::open_tuned_connection(&path)?; + let pending_conn = Self::open_recovery_connection(&path)?; pending_conn .execute( "CREATE TABLE IF NOT EXISTS shielded_pending_spends ( @@ -215,14 +449,65 @@ impl FileBackedShieldedStore { .map_err(|e| FileShieldedStoreError::Storage(format!("upgrade pending schema: {e}")))?; } } + // A proved height at which the record's anchor was recorded, when + // known (`PendingRedrive::anchor_height`): NULL for a record saved + // before it was kept. + if !columns.iter().any(|existing| existing == "anchor_height") { + pending_conn + .execute( + "ALTER TABLE shielded_pending_spends ADD COLUMN anchor_height INTEGER", + [], + ) + .map_err(|e| { + FileShieldedStoreError::Storage(format!("upgrade pending schema: {e}")) + })?; + } + // The private contact request a record pays for, when it does. + if !columns + .iter() + .any(|existing| existing == "private_contact_payment") + { + pending_conn + .execute( + "ALTER TABLE shielded_pending_spends ADD COLUMN private_contact_payment BLOB", + [], + ) + .map_err(|e| { + FileShieldedStoreError::Storage(format!("upgrade pending schema: {e}")) + })?; + } + // The private payment journal: one record per private contact request + // payment that may have reached the network, by wallet and entry + // commitment, and the wallets whose journal is open. Never purged. + pending_conn + .execute_batch( + "CREATE TABLE IF NOT EXISTS shielded_private_payment_journal ( + wallet_id BLOB NOT NULL, + commitment BLOB NOT NULL, + record BLOB NOT NULL, + PRIMARY KEY (wallet_id, commitment) + ); + CREATE TABLE IF NOT EXISTS shielded_private_payment_journal_wallets ( + wallet_id BLOB NOT NULL PRIMARY KEY + );", + ) + .map_err(|e| { + FileShieldedStoreError::Storage(format!("create private payment journal: {e}")) + })?; let mut store = Self { tree: Mutex::new(tree), path, max_checkpoints, subwallets: BTreeMap::new(), pending_conn: Mutex::new(pending_conn), + private_contact_payments: BTreeMap::new(), + unloaded_private_contact_payments: BTreeSet::new(), + private_payment_journal: BTreeMap::new(), + private_payment_journals: BTreeSet::new(), + unreadable_private_payment_journal: BTreeSet::new(), }; store.rehydrate_pending_spends()?; + store.load_private_payment_journal()?; Ok(store) } @@ -239,7 +524,8 @@ impl FileBackedShieldedStore { let mut stmt = conn .prepare( "SELECT wallet_id, account_index, activity_id, anchor, nullifiers, st_bytes, \ - attempts, identity_nonce_finalized, identity_user_abandoned FROM shielded_pending_spends", + attempts, identity_nonce_finalized, identity_user_abandoned, \ + private_contact_payment, anchor_height FROM shielded_pending_spends", ) .map_err(|e| FileShieldedStoreError::Storage(format!("prepare rehydrate: {e}")))?; let rows = stmt @@ -249,6 +535,32 @@ impl FileBackedShieldedStore { // A malformed st_bytes value remains unclassifiable and must // fail closed. let classifiable_st_bytes = row.get::<_, Vec>(5).ok(); + // The wallet, read on its own too, for a damaged record that is + // retained without being loaded. + let readable_wallet_id = row + .get::<_, Vec>(0) + .ok() + .and_then(|bytes| <[u8; 32]>::try_from(bytes.as_slice()).ok()); + // SQL NULL is a record with no association. Anything else that + // does not read back fails restoring the wallet's reservations + // rather than losing the pair it holds. + let private_contact_payment = match row.get::<_, Option>>(9) { + Ok(None) => None, + Ok(Some(bytes)) => Some( + PrivateContactPayment::from_bytes(&bytes) + .map_or(SavedPrivateContactPayment::Unreadable, |payment| { + SavedPrivateContactPayment::Readable(payment) + }), + ), + Err(_) => Some(SavedPrivateContactPayment::Unreadable), + }; + // A height that does not read is not known: the anchor is then + // not proven pruned until a set is seen holding it. + let anchor_height = row + .get::<_, Option>(10) + .ok() + .flatten() + .and_then(|height| u64::try_from(height).ok()); let decoded: rusqlite::Result<_> = (|| { Ok(( row.get::<_, Vec>(0)?, @@ -262,15 +574,26 @@ impl FileBackedShieldedStore { row.get::<_, i64>(8)?, )) })(); - Ok((classifiable_st_bytes, decoded)) + Ok(( + classifiable_st_bytes, + readable_wallet_id, + private_contact_payment, + anchor_height, + decoded, + )) }) .map_err(|e| FileShieldedStoreError::Storage(format!("query rehydrate: {e}")))?; for row in rows { - let (classifiable_st_bytes, decoded) = - row.map_err(|e| FileShieldedStoreError::RecoveryCorrupted { - account_index: None, - reason: format!("cannot decode recovery row: {e}"), - })?; + let ( + classifiable_st_bytes, + readable_wallet_id, + private_contact_payment, + anchor_height, + decoded, + ) = row.map_err(|e| FileShieldedStoreError::RecoveryCorrupted { + account_index: None, + reason: format!("cannot decode recovery row: {e}"), + })?; let note_spend_kind = classifiable_st_bytes .as_deref() .and_then(Self::non_identity_note_spend_kind); @@ -293,6 +616,11 @@ impl FileBackedShieldedStore { error = %error, "retaining but skipping a damaged shielded note-spend recovery row" ); + if private_contact_payment.is_some() || kind == PRIVATE_DOCUMENT_CREATE_KIND + { + self.unloaded_private_contact_payments + .insert(readable_wallet_id); + } continue; } return Err(FileShieldedStoreError::RecoveryCorrupted { @@ -351,6 +679,7 @@ impl FileBackedShieldedStore { nullifiers, st_bytes, attempts, + anchor_height, identity_nonce_finalized, identity_user_abandoned, }, @@ -366,15 +695,29 @@ impl FileBackedShieldedStore { reason = %error, "retaining but skipping a damaged shielded note-spend recovery row" ); + if private_contact_payment.is_some() || kind == PRIVATE_DOCUMENT_CREATE_KIND + { + self.unloaded_private_contact_payments + .insert(readable_wallet_id); + } continue; } return Err(error); } }; + if let Some(payment) = private_contact_payment { + self.private_contact_payments + .insert((id, redrive.activity_id), payment); + } let sw = self.subwallets.entry(id).or_default(); for n in &redrive.nullifiers { sw.mark_pending(n); - sw.set_pending_spend(n, redrive.anchor, redrive.activity_id); + sw.set_pending_spend( + n, + redrive.anchor, + redrive.activity_id, + redrive.anchor_height, + ); } sw.arm_redrive(redrive); } @@ -399,6 +742,9 @@ impl FileBackedShieldedStore { StateTransition::IdentityTopUpFromShieldedPool(_) => { Some("identity top up from shielded pool") } + StateTransition::PrivateDocumentCreateFromShieldedPool(_) => { + Some(PRIVATE_DOCUMENT_CREATE_KIND) + } _ => None, } } @@ -413,6 +759,21 @@ impl FileBackedShieldedStore { /// /// [`open_path`]: Self::open_path /// [`reset_commitment_tree`]: ShieldedStore::reset_commitment_tree + /// [`Self::open_tuned_connection`] committing at `synchronous=FULL`, for + /// the recovery tables: in WAL mode that syncs the WAL on every commit, + /// so a recovery record saved before a broadcast survives a crash or + /// power loss. The commitment tree's connection keeps `NORMAL`. + fn open_recovery_connection( + path: &Path, + ) -> Result { + let conn = Self::open_tuned_connection(path)?; + conn.pragma_update(None, "synchronous", "FULL") + .map_err(|e| { + FileShieldedStoreError::Storage(format!("PRAGMA synchronous=FULL: {e}")) + })?; + Ok(conn) + } + fn open_tuned_connection(path: &Path) -> Result { let conn = rusqlite::Connection::open(path) .map_err(|e| FileShieldedStoreError::Storage(format!("open sqlite: {e}")))?; @@ -565,6 +926,7 @@ impl ShieldedStore for FileBackedShieldedStore { rehydrate on the next open (self-heals via reconcile/prune)" ); } + self.private_contact_payments.remove(&(id, *activity_id)); } Ok(outcome.newly_spent) } @@ -595,20 +957,104 @@ impl ShieldedStore for FileBackedShieldedStore { rehydrate on the next open (self-heals via reconcile/prune)" ); } + self.evict_private_contact_payments(id); } Ok(removed) } + fn mark_pending_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result { + Ok(self + .subwallets + .entry(id) + .or_default() + .mark_pending_by(nullifier, reservation)) + } + + fn clear_pending_reserved_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result { + let Some(sw) = self.subwallets.get_mut(&id) else { + return Ok(false); + }; + let released = sw.clear_pending_reserved_by(nullifier, reservation); + if released { + // Same log-don't-abort rationale as `clear_pending`. + if let Err(e) = self.delete_redrive_rows_containing(id, nullifier) { + tracing::warn!( + error = %e, + "redrive row deletion failed after a released reservation; a stale row \ + may rehydrate on the next open (self-heals via reconcile/prune)" + ); + } + self.evict_private_contact_payments(id); + } + Ok(released) + } + fn set_pending_spend( &mut self, id: SubwalletId, nullifier: &[u8; 32], anchor: [u8; 32], activity_id: [u8; 32], + anchor_height: Option, ) -> Result<(), Self::Error> { if let Some(sw) = self.subwallets.get_mut(&id) { - sw.set_pending_spend(nullifier, anchor, activity_id); + sw.set_pending_spend(nullifier, anchor, activity_id, anchor_height); + } + Ok(()) + } + + fn note_anchor_recorded( + &mut self, + id: SubwalletId, + anchor: &[u8; 32], + height: u64, + ) -> Result<(), Self::Error> { + let Some(sw) = self.subwallets.get_mut(&id) else { + return Ok(()); + }; + // Durable first: a failure gives nothing a height, which holds. So + // does a height past SQLite's integer range. + let Ok(durable_height) = i64::try_from(height) else { + return Ok(()); + }; + let records = sw.redrives_without_anchor_height(anchor); + if !records.is_empty() { + let mut conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + let tx = conn.transaction().map_err(|e| { + FileShieldedStoreError::Storage(format!("begin anchor height: {e}")) + })?; + for activity_id in &records { + tx.execute( + "UPDATE shielded_pending_spends SET anchor_height = ?4 \ + WHERE wallet_id = ?1 AND account_index = ?2 AND activity_id = ?3 \ + AND anchor_height IS NULL", + rusqlite::params![ + id.wallet_id.as_slice(), + id.account_index, + activity_id.as_slice(), + durable_height, + ], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("note anchor height: {e}")))?; + } + tx.commit().map_err(|e| { + FileShieldedStoreError::Storage(format!("commit anchor height: {e}")) + })?; } + sw.note_anchor_recorded(anchor, height); Ok(()) } @@ -621,31 +1067,243 @@ impl ShieldedStore for FileBackedShieldedStore { } fn arm_redrive(&mut self, id: SubwalletId, redrive: PendingRedrive) -> Result<(), Self::Error> { + self.arm_redrive_paying_for(id, redrive, None, &[]) + } + + fn arm_private_contact_payment( + &mut self, + id: SubwalletId, + redrive: PendingRedrive, + payment: PrivateContactPayment, + commitments: &[[u8; 32]], + anchor_height: u64, + ) -> Result<(), Self::Error> { + let records: Vec = commitments + .iter() + .map(|commitment| PrivatePaymentRecord { + account_index: id.account_index, + activity_id: redrive.activity_id, + anchor: redrive.anchor, + anchor_height, + commitment: *commitment, + payment, + nullifiers: redrive.nullifiers.clone(), + }) + .collect(); + self.arm_redrive_paying_for(id, redrive, Some(payment), &records) + } + + fn private_payment_journal_open(&self, wallet_id: WalletId) -> Result { + Ok(self.private_payment_journals.contains(&wallet_id)) + } + + fn open_private_payment_journal(&mut self, wallet_id: WalletId) -> Result<(), Self::Error> { + if self.private_payment_journals.contains(&wallet_id) { + return Ok(()); + } { - let conn = self.pending_conn.lock().expect("pending_conn mutex"); - let nullifier_blob: Vec = redrive.nullifiers.iter().flatten().copied().collect(); + let conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; conn.execute( - "INSERT OR REPLACE INTO shielded_pending_spends \ - (wallet_id, account_index, activity_id, anchor, nullifiers, st_bytes, attempts, identity_nonce_finalized, identity_user_abandoned) \ - VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)", - rusqlite::params![ - id.wallet_id.as_slice(), - id.account_index, - redrive.activity_id.as_slice(), - redrive.anchor.as_slice(), - nullifier_blob, - redrive.st_bytes, - redrive.attempts, - redrive.identity_nonce_finalized, - redrive.identity_user_abandoned, - ], + "INSERT OR IGNORE INTO shielded_private_payment_journal_wallets (wallet_id) \ + VALUES (?1)", + rusqlite::params![wallet_id.as_slice()], ) - .map_err(|e| FileShieldedStoreError::Storage(format!("persist redrive: {e}")))?; + .map_err(|e| FileShieldedStoreError::Storage(format!("open payment journal: {e}")))?; + } + self.private_payment_journals.insert(wallet_id); + Ok(()) + } + + /// Fails with [`FileShieldedStoreError::RecoveryCorrupted`] when a + /// journal record of the wallet, or one whose wallet does not read, does + /// not read back: the pair it holds could not be restored. + fn private_payment_journal( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error> { + if self + .unreadable_private_payment_journal + .contains(&Some(wallet_id)) + || self.unreadable_private_payment_journal.contains(&None) + { + return Err(FileShieldedStoreError::RecoveryCorrupted { + account_index: None, + reason: "a private payment journal record of the wallet does not read back" + .to_string(), + }); + } + Ok(self + .private_payment_journal + .range((wallet_id, [0u8; 32])..=(wallet_id, [u8::MAX; 32])) + .map(|(_, record)| record.clone()) + .collect()) + } + + fn journal_private_payments( + &mut self, + wallet_id: WalletId, + records: &[PrivatePaymentRecord], + ) -> Result<(), Self::Error> { + let rows = Self::journal_rows(records)?; + { + let mut conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + let tx = conn + .transaction() + .map_err(|e| FileShieldedStoreError::Storage(format!("begin journal: {e}")))?; + Self::insert_journal_rows(&tx, wallet_id, &rows)?; + tx.commit() + .map_err(|e| FileShieldedStoreError::Storage(format!("commit journal: {e}")))?; + } + if !records.is_empty() { + self.private_payment_journals.insert(wallet_id); + } + for record in records { + self.private_payment_journal + .insert((wallet_id, record.commitment), record.clone()); } - self.subwallets.entry(id).or_default().arm_redrive(redrive); Ok(()) } + fn retire_undelivered_spend( + &mut self, + id: SubwalletId, + activity_id: &[u8; 32], + commitments: &[[u8; 32]], + ) -> Result { + if self + .subwallets + .get(&id) + .and_then(|subwallet| subwallet.redrive_attempts(activity_id)) + != Some(0) + { + return Ok(false); + } + // Durable first, in one transaction: a failure removes nothing, so the + // record and the journal records stay together, which over-holds. + { + let mut conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + let tx = conn + .transaction() + .map_err(|e| FileShieldedStoreError::Storage(format!("begin retirement: {e}")))?; + // Only a record no re-broadcast was claimed for: the durable + // counter is written before memory's. + let retired = tx + .execute( + "DELETE FROM shielded_pending_spends \ + WHERE wallet_id = ?1 AND account_index = ?2 AND activity_id = ?3 \ + AND attempts = 0", + rusqlite::params![ + id.wallet_id.as_slice(), + id.account_index, + activity_id.as_slice(), + ], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("retire redrive: {e}")))?; + if retired != 1 { + // Dropped uncommitted: nothing is removed. + return Ok(false); + } + for commitment in commitments { + tx.execute( + "DELETE FROM shielded_private_payment_journal \ + WHERE wallet_id = ?1 AND commitment = ?2", + rusqlite::params![id.wallet_id.as_slice(), commitment.as_slice()], + ) + .map_err(|e| { + FileShieldedStoreError::Storage(format!("discharge retired payment: {e}")) + })?; + } + tx.commit() + .map_err(|e| FileShieldedStoreError::Storage(format!("commit retirement: {e}")))?; + } + if let Some(subwallet) = self.subwallets.get_mut(&id) { + subwallet.clear_redrive(activity_id); + } + self.private_contact_payments.remove(&(id, *activity_id)); + for commitment in commitments { + self.private_payment_journal + .remove(&(id.wallet_id, *commitment)); + } + Ok(true) + } + + fn discharge_private_payment( + &mut self, + wallet_id: WalletId, + commitment: &[u8; 32], + ) -> Result { + // Durable first: a failure leaves the record, which over-holds. + { + let conn = self + .pending_conn + .lock() + .map_err(|e| FileShieldedStoreError::Storage(format!("pending_conn mutex: {e}")))?; + conn.execute( + "DELETE FROM shielded_private_payment_journal \ + WHERE wallet_id = ?1 AND commitment = ?2", + rusqlite::params![wallet_id.as_slice(), commitment.as_slice()], + ) + .map_err(|e| FileShieldedStoreError::Storage(format!("discharge payment: {e}")))?; + } + Ok(self + .private_payment_journal + .remove(&(wallet_id, *commitment)) + .is_some()) + } + + /// Fails with [`FileShieldedStoreError::RecoveryCorrupted`] when a live + /// record of the wallet carries an association that does not read back, + /// or a damaged record of it retained without being loaded carries one: + /// the pair that payment holds could not be restored, so the restoration + /// fails and the wallet's holds stay. + fn private_contact_redrives_for_wallet( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error> { + if self + .unloaded_private_contact_payments + .contains(&Some(wallet_id)) + || self.unloaded_private_contact_payments.contains(&None) + { + return Err(FileShieldedStoreError::RecoveryCorrupted { + account_index: None, + reason: "a damaged saved payment of the wallet carries a private contact request \ + association" + .to_string(), + }); + } + let mut payments = Vec::new(); + for (id, redrive) in self.pending_redrives_for_wallet(wallet_id)? { + match self + .private_contact_payments + .get(&(id, redrive.activity_id)) + { + None => {} + Some(SavedPrivateContactPayment::Readable(payment)) => { + payments.push((redrive, *payment)); + } + Some(SavedPrivateContactPayment::Unreadable) => { + return Err(FileShieldedStoreError::RecoveryCorrupted { + account_index: Some(id.account_index), + reason: "the private contact request a saved payment pays for does not \ + decode" + .to_string(), + }); + } + } + } + Ok(payments) + } + fn pending_redrives(&self, id: SubwalletId) -> Result, Self::Error> { Ok(self .subwallets @@ -720,6 +1378,7 @@ impl ShieldedStore for FileBackedShieldedStore { if let Some(sw) = self.subwallets.get_mut(&id) { sw.clear_redrive(activity_id); } + self.private_contact_payments.remove(&(id, *activity_id)); Ok(()) } @@ -899,6 +1558,10 @@ impl ShieldedStore for FileBackedShieldedStore { // in-memory only (`subwallets`); the commitment tree in // SQLite is chain-wide and intentionally left intact. self.subwallets.retain(|id, _| id.wallet_id != wallet_id); + self.private_contact_payments + .retain(|(id, _), _| id.wallet_id != wallet_id); + self.unloaded_private_contact_payments + .remove(&Some(wallet_id)); Ok(()) } @@ -931,6 +1594,7 @@ impl ShieldedStore for FileBackedShieldedStore { retained.arm_redrive(redrive); } } + self.evict_private_contact_payments(id); Ok(()) } @@ -946,6 +1610,8 @@ impl ShieldedStore for FileBackedShieldedStore { })?; } self.subwallets.clear(); + self.private_contact_payments.clear(); + self.unloaded_private_contact_payments.clear(); Ok(()) } @@ -1014,6 +1680,7 @@ impl ShieldedStore for FileBackedShieldedStore { #[cfg(test)] mod tests { use super::*; + use dpp::state_transition::shielded_transfer_transition::v0::ShieldedTransferTransitionV0; /// Unique temp path for a test tree (no `tempfile` dev-dep). fn temp_tree_path(tag: &str) -> std::path::PathBuf { @@ -1039,6 +1706,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); @@ -1083,6 +1751,156 @@ mod tests { let _ = std::fs::remove_file(&path); } + /// The private contact request a record pays for is saved with it: it + /// survives a reopen and an attempt bump, is listed only joined with its + /// live record, and goes with the record. A record armed without one is + /// not listed. + #[test] + fn should_keep_a_private_contact_payment_with_its_record_across_reopen() { + let path = temp_tree_path("private_contact_payment"); + let wallet_id = [0x5A; 32]; + let id = SubwalletId::new(wallet_id, 0); + let record = |activity: u8, nullifier: u8| PendingRedrive { + activity_id: [activity; 32], + anchor: [2u8; 32], + nullifiers: vec![[nullifier; 32]], + st_bytes: vec![0xAB; 96], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 0x1000_0005, + }; + { + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + store + .arm_private_contact_payment(id, record(1, 3), payment, &[[0xC1; 32]], 0) + .expect("arm the payment"); + store.arm_redrive(id, record(4, 6)).expect("arm a spend"); + store.bump_redrive_attempts(id, &[1u8; 32]).expect("bump"); + } + { + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("reopen"); + let listed = store + .private_contact_redrives_for_wallet(wallet_id) + .expect("listed"); + assert_eq!(listed.len(), 1, "only the payment: {listed:?}"); + assert_eq!(listed[0].0.activity_id, [1u8; 32]); + assert_eq!(listed[0].0.attempts, 1); + assert_eq!(listed[0].1, payment); + assert!(store + .private_contact_redrives_for_wallet([0x5B; 32]) + .expect("another wallet") + .is_empty()); + assert!(store.clear_pending(id, &[3u8; 32]).expect("clear")); + assert!(store + .private_contact_redrives_for_wallet(wallet_id) + .expect("listed") + .is_empty()); + assert!( + store.private_contact_payments.is_empty(), + "the association goes with its record" + ); + } + { + let store = FileBackedShieldedStore::open_path(&path, 100).expect("reopen 2"); + assert!(store + .private_contact_redrives_for_wallet(wallet_id) + .expect("listed") + .is_empty()); + assert_eq!(store.pending_redrives(id).expect("redrives").len(), 1); + } + let _ = std::fs::remove_file(&path); + } + + /// A private contact request association goes with its record on every + /// path that drops the record: a resolved spend, a cleared redrive, and + /// the purges. + #[test] + fn should_drop_a_private_contact_payment_with_its_record() { + let path = temp_tree_path("private_contact_payment_eviction"); + let wallet_id = [0x5C; 32]; + let id = SubwalletId::new(wallet_id, 0); + let other = SubwalletId::new([0x5D; 32], 0); + let record = |activity: u8, nullifier: u8| PendingRedrive { + activity_id: [activity; 32], + anchor: [2u8; 32], + nullifiers: vec![[nullifier; 32]], + st_bytes: vec![0xAB; 96], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }; + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + let arm = |store: &mut FileBackedShieldedStore, id, activity, nullifier| { + store + .arm_private_contact_payment( + id, + record(activity, nullifier), + payment, + &[[activity; 32]], + 0, + ) + .expect("armed"); + }; + + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [11; 32], + block_height: 1, + is_spent: false, + value: 1, + note_data: vec![], + }; + store.save_note(id, ¬e).expect("a note"); + arm(&mut store, id, 1, 11); + store.mark_spent(id, &[11; 32]).expect("spent"); + assert!(store.private_contact_payments.is_empty(), "resolved spend"); + + arm(&mut store, id, 2, 12); + store.clear_redrive(id, &[2; 32]).expect("cleared"); + assert!(store.private_contact_payments.is_empty(), "cleared redrive"); + + arm(&mut store, id, 3, 13); + arm(&mut store, other, 4, 14); + store.purge_subwallet(id).expect("purged"); + assert_eq!( + store + .private_contact_payments + .keys() + .copied() + .collect::>(), + vec![(other, [4; 32])], + "subwallet purge" + ); + arm(&mut store, id, 5, 15); + store.purge_wallet(wallet_id).expect("purged"); + assert_eq!( + store + .private_contact_payments + .keys() + .copied() + .collect::>(), + vec![(other, [4; 32])], + "wallet purge" + ); + store.purge_all_subwallets().expect("purged"); + assert!(store.private_contact_payments.is_empty(), "every subwallet"); + drop(store); + let _ = std::fs::remove_file(&path); + } + #[test] fn identity_debit_guard_with_no_nullifiers_survives_reopen() { let path = temp_tree_path("identity_debit_guard"); @@ -1095,6 +1913,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); @@ -1192,6 +2011,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -1221,6 +2041,105 @@ mod tests { } } + /// A saved payment's private contact request association that does not + /// read back fails restoring its wallet's reservations, rather than losing + /// the pair it holds: a non-null value that does not decode on a record + /// that loads, which is kept, and one carried by a damaged note-spend + /// record retained without being loaded. SQL NULL stays a record with no + /// association, and the wallet restores again once the record goes. + #[test] + fn should_fail_restoring_a_wallet_whose_saved_association_does_not_read() { + let path = temp_tree_path("unreadable_private_contact_payment"); + let wallet_id = [0x6A; 32]; + let id = SubwalletId::new(wallet_id, 0); + let note_transition: StateTransition = ShieldedTransferTransitionV0 { + actions: vec![], + value_balance: 1_000, + anchor: [0x52; 32], + proof: vec![], + binding_signature: [0; 64], + } + .into(); + let record = PendingRedrive { + activity_id: [0x11; 32], + anchor: [0x52; 32], + nullifiers: vec![[0x13; 32]], + st_bytes: note_transition.serialize_to_bytes().unwrap(), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }; + let arm = || { + let mut store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); + store + .arm_private_contact_payment(id, record.clone(), payment, &[[0xC2; 32]], 0) + .unwrap(); + }; + let set = |assignment: &str| { + rusqlite::Connection::open(&path) + .unwrap() + .execute( + &format!("UPDATE shielded_pending_spends SET {assignment}"), + [], + ) + .unwrap(); + }; + let unreadable = |store: &FileBackedShieldedStore| { + matches!( + store.private_contact_redrives_for_wallet(wallet_id), + Err(FileShieldedStoreError::RecoveryCorrupted { .. }) + ) + }; + + arm(); + set("private_contact_payment = x'01'"); + let mut store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); + assert!(unreadable(&store), "an association that does not decode"); + assert_eq!(store.pending_redrives(id).unwrap(), vec![record.clone()]); + assert!(store + .private_contact_redrives_for_wallet([0x6B; 32]) + .unwrap() + .is_empty()); + store.clear_redrive(id, &record.activity_id).unwrap(); + assert!( + store + .private_contact_redrives_for_wallet(wallet_id) + .unwrap() + .is_empty(), + "the record went" + ); + drop(store); + + arm(); + set("anchor = x'01'"); + let store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); + assert!(store.pending_redrives(id).unwrap().is_empty(), "not loaded"); + assert!(unreadable(&store), "a damaged record carrying one"); + drop(store); + + set( + "anchor = x'5252525252525252525252525252525252525252525252525252525252525252', \ + private_contact_payment = NULL", + ); + let store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); + assert_eq!(store.pending_redrives(id).unwrap(), vec![record]); + assert!( + store + .private_contact_redrives_for_wallet(wallet_id) + .unwrap() + .is_empty(), + "no association" + ); + drop(store); + let _ = std::fs::remove_file(path); + } + #[test] fn should_skip_only_damaged_rows_proven_to_spend_shielded_notes() { use dpp::state_transition::shielded_transfer_transition::v0::ShieldedTransferTransitionV0; @@ -1255,6 +2174,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); @@ -1269,6 +2189,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -1334,6 +2255,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -1387,6 +2309,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -1420,6 +2343,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).unwrap(); @@ -1447,6 +2371,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); @@ -1496,6 +2421,7 @@ mod tests { attempts: 5, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; let note_spend = PendingRedrive { activity_id: [0x48; 32], @@ -1505,6 +2431,7 @@ mod tests { attempts: 1, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); @@ -1550,6 +2477,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; // purge_wallet is scoped: it drops A's rows, keeps B's. @@ -1627,6 +2555,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .expect("arm"); @@ -1932,7 +2861,7 @@ mod tests { store.checkpoint_tree(4).unwrap(); let wallet_depth0_mid_block = store.tree_anchor().unwrap(); - // The spend path uses exactly this anchor: `extract_spends_and_anchor` + // The spend path uses exactly this anchor: `extract_spends_and_recorded_anchor` // builds it as `witness(pos, 0).root(cmx)`. Pin that it equals the // mid-block `tree_anchor()`. let cmx0 = ExtractedNoteCommitment::from_bytes(&cmx(1)) @@ -1974,4 +2903,345 @@ mod tests { recorded set is exactly these two and the mid-block anchor is outside it" ); } + + /// A type 29 transition paying for one entry whose commitment is + /// `[commitment; 32]`, with placeholder actions, serialized. + fn private_document_create_bytes(commitment: u8) -> Vec { + use dpp::platform_value::BinaryData; + use dpp::shielded::SerializedAction; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::methods::PrivateDocumentCreateFromShieldedPoolTransitionMethodsV0; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::PrivateDocumentCreateFromShieldedPoolTransition; + use dpp::version::PlatformVersion; + + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + let mut entry = vec![commitment; 32]; + entry.extend_from_slice(&[0; 557]); + PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + dpp::prelude::Identifier::from([0x44; 32]), + "privateContactRequest".to_string(), + vec![BinaryData::new(entry)], + vec![action.clone(), action], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition") + .serialize_to_bytes() + .expect("serializes") + } + + /// The recovery tables commit at `synchronous=FULL`, so a record saved + /// before a broadcast survives a power loss; the commitment tree's + /// connection keeps `NORMAL`. + #[test] + fn should_commit_recovery_records_at_full_sync() { + let path = temp_tree_path("recovery_full_sync"); + let store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + let synchronous = |conn: &rusqlite::Connection| { + conn.pragma_query_value(None, "synchronous", |row| row.get::<_, i64>(0)) + .expect("synchronous") + }; + let recovery = store.pending_conn.lock().expect("pending_conn mutex"); + assert_eq!(synchronous(&recovery), 2, "FULL"); + drop(recovery); + let tree = FileBackedShieldedStore::open_tuned_connection(&path).expect("tree conn"); + assert_eq!(synchronous(&tree), 1, "NORMAL"); + let _ = std::fs::remove_file(&path); + } + + /// A journal record is saved with its payment's redrive record, and + /// outlives it: no note release, purge or reopen removes it, only a + /// discharge does. The journal it was saved in reads as open. + #[test] + fn should_keep_a_journal_record_until_it_is_discharged() { + let path = temp_tree_path("journal_lifecycle"); + let wallet_id = [0x5A; 32]; + let id = SubwalletId::new(wallet_id, 1); + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 7, + }; + let redrive = PendingRedrive { + activity_id: [0x21; 32], + anchor: [0x22; 32], + nullifiers: vec![[0x23; 32], [0x24; 32]], + st_bytes: private_document_create_bytes(0xC3), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + let expected = PrivatePaymentRecord { + account_index: 1, + activity_id: redrive.activity_id, + anchor: redrive.anchor, + anchor_height: 0, + commitment: [0xC3; 32], + payment, + nullifiers: redrive.nullifiers.clone(), + }; + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + assert!(!store + .private_payment_journal_open(wallet_id) + .expect("open?")); + for nullifier in &redrive.nullifiers { + store.mark_pending(id, nullifier).expect("reserved"); + } + store + .arm_private_contact_payment(id, redrive.clone(), payment, &[[0xC3; 32]], 0) + .expect("armed"); + assert!(store + .private_payment_journal_open(wallet_id) + .expect("open?")); + assert_eq!( + store.private_payment_journal(wallet_id).expect("journal"), + vec![expected.clone()] + ); + // A note release drops the redrive record, not the journal record. + assert!(store.clear_pending(id, &[0x23; 32]).expect("released")); + assert!(store.pending_redrives(id).expect("records").is_empty()); + store.purge_subwallet(id).expect("account purge"); + store.purge_wallet(wallet_id).expect("wallet purge"); + store.purge_all_subwallets().expect("purge all"); + drop(store); + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("reopened"); + assert!(store + .private_payment_journal_open(wallet_id) + .expect("open?")); + assert_eq!( + store.private_payment_journal(wallet_id).expect("journal"), + vec![expected] + ); + assert!(store + .private_payment_journal([0x5B; 32]) + .expect("another wallet's") + .is_empty()); + assert!(store + .discharge_private_payment(wallet_id, &[0xC3; 32]) + .expect("discharged")); + drop(store); + let store = FileBackedShieldedStore::open_path(&path, 100).expect("reopened"); + assert!(store + .private_payment_journal(wallet_id) + .expect("journal") + .is_empty()); + let _ = std::fs::remove_file(&path); + } + + /// A never-delivered spend's saved record and its journal records are + /// retired by one durable write: after it, a reopened store holds + /// neither. A write that fails part way, here at the journal's delete, + /// removes neither, and one whose re-broadcast was claimed is refused and + /// keeps both; so after any reopen, the transition the redrive pass could + /// broadcast and the journal record that holds its pair stay together. + #[test] + fn should_retire_an_undelivered_spend_with_its_journal_records_in_one_write() { + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 7, + }; + let commitment = [0xC4; 32]; + for case in ["retired", "write fails", "re-broadcast claimed"] { + let path = temp_tree_path("retire_undelivered"); + let wallet_id = [0x5C; 32]; + let id = SubwalletId::new(wallet_id, 0); + let redrive = PendingRedrive { + activity_id: [0x31; 32], + anchor: [0x32; 32], + nullifiers: vec![[0x33; 32]], + st_bytes: private_document_create_bytes(0xC4), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + store.mark_pending(id, &[0x33; 32]).expect("reserved"); + store + .arm_private_contact_payment(id, redrive.clone(), payment, &[commitment], 0) + .expect("armed"); + match case { + "write fails" => store + .pending_conn + .lock() + .expect("pending_conn mutex") + .execute_batch( + "CREATE TRIGGER reject_journal_delete BEFORE DELETE ON \ + shielded_private_payment_journal BEGIN SELECT RAISE(FAIL, \ + 'injected delete failure'); END;", + ) + .expect("trigger"), + "re-broadcast claimed" => { + assert_eq!( + store + .bump_redrive_attempts(id, &redrive.activity_id) + .expect("claimed"), + 1 + ); + } + _ => {} + } + let retired = store.retire_undelivered_spend(id, &redrive.activity_id, &[commitment]); + match case { + "retired" => assert!(retired.expect("retired"), "{case}"), + "write fails" => assert!(retired.is_err(), "{case}: {retired:?}"), + _ => assert!(!retired.expect("refused"), "{case}"), + } + let kept = case != "retired"; + let both = |store: &FileBackedShieldedStore| { + ( + store.pending_redrives(id).expect("records").len(), + store + .private_payment_journal(wallet_id) + .expect("journal") + .len(), + ) + }; + let expected = if kept { (1, 1) } else { (0, 0) }; + assert_eq!(both(&store), expected, "{case}: in memory"); + drop(store); + let reopened = FileBackedShieldedStore::open_path(&path, 100).expect("reopened"); + assert_eq!(both(&reopened), expected, "{case}: reopened"); + drop(reopened); + let _ = std::fs::remove_file(&path); + } + } + + /// A journal record that does not read back stays, and fails reading + /// its wallet's journal, or every wallet's when its wallet does not read + /// either; the store still opens. + #[test] + fn should_fail_reading_a_journal_whose_record_does_not_read() { + for (damage, other_wallet_reads) in [ + ( + "UPDATE shielded_private_payment_journal SET record = x'01'", + true, + ), + ( + "UPDATE shielded_private_payment_journal SET commitment = x'02'", + true, + ), + ( + "UPDATE shielded_private_payment_journal SET wallet_id = x'03'", + false, + ), + ] { + let path = temp_tree_path("journal_damage"); + let wallet_id = [0x6A; 32]; + let id = SubwalletId::new(wallet_id, 0); + let payment = PrivateContactPayment { + sender: [0xB2; 32], + recipient: [0xA2; 32], + account_reference: 1, + }; + { + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + store + .arm_private_contact_payment( + id, + PendingRedrive { + activity_id: [0x31; 32], + anchor: [0x32; 32], + nullifiers: vec![[0x33; 32]], + st_bytes: private_document_create_bytes(0xD4), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + payment, + &[[0xD4; 32]], + 0, + ) + .expect("armed"); + store + .pending_conn + .lock() + .expect("pending_conn mutex") + .execute(damage, []) + .expect("damaged"); + } + let store = FileBackedShieldedStore::open_path(&path, 100).expect("still opens"); + assert!( + matches!( + store.private_payment_journal(wallet_id), + Err(FileShieldedStoreError::RecoveryCorrupted { .. }) + ), + "{damage}" + ); + assert_eq!( + store.private_payment_journal([0x6B; 32]).is_ok(), + other_wallet_reads, + "{damage}" + ); + let rows: i64 = store + .pending_conn + .lock() + .expect("pending_conn mutex") + .query_row( + "SELECT count(*) FROM shielded_private_payment_journal", + [], + |row| row.get(0), + ) + .expect("count"); + assert_eq!(rows, 1, "{damage}: the record stays"); + let _ = std::fs::remove_file(&path); + } + } + + /// A private document create (type 29) is a note spend: a recovery row + /// for one with damaged metadata is retained and skipped, its wallet + /// flagged so its legacy payments do not restore, and the shared store + /// still opens for every wallet. + #[test] + fn should_retain_and_skip_a_damaged_private_document_payment_row() { + let path = temp_tree_path("damaged_private_document_row"); + let wallet_id = [0x7A; 32]; + let id = SubwalletId::new(wallet_id, 0); + { + let mut store = FileBackedShieldedStore::open_path(&path, 100).expect("open"); + store + .arm_redrive( + id, + PendingRedrive { + activity_id: [0x41; 32], + anchor: [0x42; 32], + nullifiers: vec![[0x43; 32]], + st_bytes: private_document_create_bytes(0xE5), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + ) + .expect("armed"); + store + .pending_conn + .lock() + .expect("pending_conn mutex") + .execute("UPDATE shielded_pending_spends SET anchor = x'01'", []) + .expect("damaged"); + } + let store = FileBackedShieldedStore::open_path(&path, 100).expect("still opens"); + assert!(store.pending_redrives(id).expect("records").is_empty()); + assert!(matches!( + store.private_contact_redrives_for_wallet(wallet_id), + Err(FileShieldedStoreError::RecoveryCorrupted { .. }) + )); + assert!(store + .private_contact_redrives_for_wallet([0x7B; 32]) + .is_ok()); + let _ = std::fs::remove_file(&path); + } } diff --git a/packages/rs-platform-wallet/src/wallet/shielded/mod.rs b/packages/rs-platform-wallet/src/wallet/shielded/mod.rs index e5c92c28be6..db547420497 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/mod.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/mod.rs @@ -70,8 +70,9 @@ pub use keys::{AccountViewingKeys, OrchardKeySet}; pub use prover::CachedOrchardProver; pub use seed_pool::{SeedPoolOutcome, SeedPoolProgress, DEFAULT_SEED_POOL_TARGET_NOTES}; pub use store::{ - IdentityDebitRecoveryRecord, IdentityDebitRecoveryStatus, InMemoryShieldedStore, ShieldedNote, - ShieldedOutgoingNote, ShieldedStore, SubwalletId, + IdentityDebitRecoveryRecord, IdentityDebitRecoveryStatus, InMemoryShieldedStore, + PrivateContactPayment, PrivatePaymentRecord, ShieldedNote, ShieldedOutgoingNote, ShieldedStore, + SubwalletId, }; pub use sync::{ShieldedSyncSummary, SyncNotesResult}; diff --git a/packages/rs-platform-wallet/src/wallet/shielded/note_selection.rs b/packages/rs-platform-wallet/src/wallet/shielded/note_selection.rs index d6fee1d7ba8..86f1b1fe593 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/note_selection.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/note_selection.rs @@ -8,9 +8,10 @@ use super::store::ShieldedNote; use crate::error::PlatformWalletError; use dpp::fee::Credits; use dpp::shielded::{ - compute_minimum_shielded_fee, compute_shielded_identity_create_fee, - compute_shielded_identity_top_up_fee, compute_shielded_unshield_fee, - compute_shielded_withdrawal_fee, is_allowed_identity_create_denomination, + compute_minimum_shielded_fee, compute_private_document_create_from_shielded_pool_fee, + compute_shielded_identity_create_fee, compute_shielded_identity_top_up_fee, + compute_shielded_unshield_fee, compute_shielded_withdrawal_fee, + is_allowed_identity_create_denomination, }; use dpp::version::PlatformVersion; use dpp::ProtocolError; @@ -50,6 +51,15 @@ pub enum ShieldedFeeKind { /// `compute_shielded_identity_top_up_fee`: IdentityTopUpFromShieldedPool (base plus the flat /// identity-balance write cost, the Unshield model with an identity as the output). IdentityTopUp, + /// `compute_private_document_create_from_shielded_pool_fee`: + /// PrivateDocumentCreateFromShieldedPool (base plus, per entry, its storage and the flat + /// processing fee). Nothing leaves the pool but the fee, so the amount selected for is 0. + PrivateDocumentCreate { + /// How many entries the transition appends. + entry_count: usize, + /// The bytes of each, the private type's `entrySize`. + entry_size: u32, + }, } impl ShieldedFeeKind { @@ -73,6 +83,15 @@ impl ShieldedFeeKind { ShieldedFeeKind::IdentityTopUp => { compute_shielded_identity_top_up_fee(num_actions, platform_version) } + ShieldedFeeKind::PrivateDocumentCreate { + entry_count, + entry_size, + } => compute_private_document_create_from_shielded_pool_fee( + num_actions, + entry_count, + entry_size, + platform_version, + ), } } } @@ -335,6 +354,34 @@ mod tests { assert!(total >= denomination); } + /// A private document create selects notes for its fee alone, the fee consensus pins for + /// its action count and its entries' number and size: one DashPay entry from two actions is + /// the 114,140,000-credit base plus the entry's storage and its processing fee. + #[test] + fn test_select_for_a_private_document_create_covers_its_fee_alone() { + let platform_version = PlatformVersion::latest(); + let kind = ShieldedFeeKind::PrivateDocumentCreate { + entry_count: 1, + entry_size: 589, + }; + let fee = kind.compute(2, platform_version).expect("the fee"); + assert_eq!( + fee, + compute_private_document_create_from_shielded_pool_fee(2, 1, 589, platform_version) + .expect("the consensus fee") + ); + assert_eq!(fee, 114_140_000 + (589 + 32) * 27_400 + 10_000_000); + + let notes = vec![test_note(fee, 0), test_note(fee - 1, 1)]; + let (selected, total, exact_fee) = + select_notes_with_fee(¬es, 0, 2, kind, platform_version).expect("covers the fee"); + assert_eq!((selected.len(), total, exact_fee), (1, fee, fee)); + assert!(matches!( + select_notes_with_fee(¬es[1..], 0, 2, kind, platform_version), + Err(PlatformWalletError::ShieldedInsufficientBalance { .. }) + )); + } + #[test] fn test_select_needs_multiple() { let notes = vec![test_note(100, 0), test_note(200, 1), test_note(150, 2)]; diff --git a/packages/rs-platform-wallet/src/wallet/shielded/operations.rs b/packages/rs-platform-wallet/src/wallet/shielded/operations.rs index 8a8f8a4b39b..bd8a0fb9eba 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/operations.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/operations.rs @@ -28,14 +28,22 @@ use super::note_selection::{ ensure_allowed_denomination, ensure_positive_amount, select_notes_for_denomination, select_notes_with_fee, ShieldedFeeKind, }; -use super::store::{PendingRedrive, ShieldedNote, ShieldedStore, SubwalletId}; -use crate::broadcast_outcome::{broadcast_definitely_failed, carries_consensus_rejection}; +use super::store::{ + PendingRedrive, PrivateContactPayment, ShieldedNote, ShieldedStore, SubwalletId, +}; +use crate::broadcast_outcome::{ + broadcast_definitely_failed, broadcast_tracking_delivery, carries_consensus_rejection, + BroadcastDelivery, +}; use crate::changeset::{PlatformWalletChangeSet, ShieldedChangeSet}; use crate::error::{preserve_signer_key_unavailable_or, PlatformWalletError}; +use crate::wallet::identity::network::PaymentStarted; use crate::wallet::persister::WalletPersister; use crate::wallet::platform_wallet::WalletId; use std::collections::{BTreeMap, HashSet}; +use std::future::Future; +use std::sync::atomic::{AtomicU64, Ordering as AtomicOrdering}; use std::sync::Arc; use dash_sdk::dapi_grpc::platform::v0::ResponseMetadata; @@ -57,12 +65,15 @@ use dpp::identity::identity_nonce::{ use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; use dpp::identity::signer::Signer; use dpp::identity::{Identity, IdentityPublicKey}; +use dpp::platform_value::BinaryData; use dpp::prelude::{Identifier, IdentityNonce}; use dpp::shielded::builder::{ build_identity_create_from_shielded_pool_transition, - build_identity_top_up_from_shielded_pool_transition, build_shield_from_identity_transition, - build_shield_transition, build_shielded_transfer_transition, - build_shielded_withdrawal_transition, build_unshield_transition, OrchardProver, SpendableNote, + build_identity_top_up_from_shielded_pool_transition, + build_private_document_create_from_shielded_pool_transition, + build_shield_from_identity_transition, build_shield_transition, + build_shielded_transfer_transition, build_shielded_withdrawal_transition, + build_unshield_transition, OrchardProver, SpendableNote, }; use dpp::shielded::compute_minimum_shielded_fee; use dpp::state_transition::proof_result::StateTransitionProofResult; @@ -278,6 +289,7 @@ fn shielded_actions(st: &StateTransition) -> &[dpp::shielded::SerializedAction] use dpp::state_transition::identity_top_up_from_shielded_pool_transition::accessors::IdentityTopUpFromShieldedPoolTransitionAccessorsV0; use dpp::state_transition::shield_from_identity_transition::accessors::ShieldFromIdentityTransitionAccessorsV0; use dpp::state_transition::unshield_transition::accessors::UnshieldTransitionAccessorsV0; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::accessors::PrivateDocumentCreateFromShieldedPoolTransitionAccessorsV0; match st { StateTransition::Shield(ShieldTransition::V0(v0)) => &v0.actions, @@ -288,6 +300,7 @@ fn shielded_actions(st: &StateTransition) -> &[dpp::shielded::SerializedAction] StateTransition::IdentityCreateFromShieldedPool(t) => t.actions(), StateTransition::ShieldFromIdentity(t) => t.actions(), StateTransition::IdentityTopUpFromShieldedPool(t) => t.actions(), + StateTransition::PrivateDocumentCreateFromShieldedPool(t) => t.actions(), _ => &[], } } @@ -881,6 +894,7 @@ async fn arm_identity_shield_redrive( attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; @@ -1399,7 +1413,7 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< let change_addr = default_orchard_address(&views)?; let id = SubwalletId::new(wallet_id, account); - let (selected_notes, total_input, exact_fee) = + let (selected_notes, total_input, exact_fee, mut reservation) = reserve_unspent_notes(sdk, store, id, amount, 2, ShieldedFeeKind::IdentityTopUp).await?; info!( @@ -1414,7 +1428,8 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< let mut pending_entry = None; let result = async { - let (spends, anchor) = extract_spends_and_anchor(sdk, store, &selected_notes).await?; + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; let anchor_bytes = anchor.to_bytes(); let (state_transition, fee_used) = build_identity_top_up_from_shielded_pool_transition( @@ -1455,7 +1470,19 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< }, ) .await; - arm_pending_release(store, id, anchor_bytes, &pending_entry, &selected_notes).await; + if arm_pending_release( + store, + id, + anchor_bytes, + anchor_height, + &pending_entry, + &selected_notes, + ) + .await? + { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } trace!("IdentityTopUpFromShieldedPool: state transition built, broadcasting..."); // Type 22 is classified affected state: its nullifiers are shared with any @@ -1470,6 +1497,7 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< id, &pending_entry, anchor_bytes, + anchor_height, &selected_notes, &state_transition, "identity top up from shielded pool", @@ -1481,6 +1509,8 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< match result { Ok((proof, metadata)) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); record_activity_status( store, persister, @@ -1535,8 +1565,15 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< }; Ok((proven_balance, metadata)) } - Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => Err(e), + Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); + Err(e) + } Err(e) => { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); record_activity_status( store, persister, @@ -1547,9 +1584,406 @@ pub(in crate::wallet) async fn identity_top_up_from_pool_with_metadata< None, ) .await; - cancel_pending(store, id, &selected_notes).await; + reservation.cancel().await; + Err(e) + } + } +} + +// ------------------------------------------------------------------------- +// PrivateDocumentCreateFromShieldedPool: shielded pool -> entries of a +// private document type's store (Type 29) +// ------------------------------------------------------------------------- + +/// Append `entries` to the store of the private document type +/// `document_type_name` of `data_contract_id` (`PrivateDocumentCreateFromShieldedPool`, +/// type 29), paying the fee from `account`'s notes, the change going back to +/// the account. Nothing but the fee leaves the pool. Every entry must take the +/// type's `entrySize`, as `platform_encryption::seal_private_document_entry` +/// builds them. +/// +/// Waits for the spent nullifiers to be proven before marking the notes +/// spent, and returns the metadata of that proof. `payment_started` is +/// marked right before the transition is broadcast. +#[allow(clippy::too_many_arguments)] +pub(in crate::wallet) async fn private_document_create_from_pool_with_metadata< + S: ShieldedStore, + P: OrchardProver, +>( + sdk: &Arc, + store: &Arc>, + persister: Option<&WalletPersister>, + wallet_id: WalletId, + keys: &OrchardKeySet, + account: u32, + data_contract_id: Identifier, + document_type_name: &str, + entries: Vec, + private_contact: Option, + payment_started: &PaymentStarted, + prover: &P, +) -> Result { + let views = keys.viewing_keys(); + let change_addr = default_orchard_address(&views)?; + let id = SubwalletId::new(wallet_id, account); + let fee_kind = ShieldedFeeKind::PrivateDocumentCreate { + entry_count: entries.len(), + entry_size: entries.first().map_or(0, |entry| entry.len() as u32), + }; + + let (selected_notes, total_input, exact_fee, reservation) = + reserve_notes_covering(sdk, store, id, 0, 2, fee_kind).await?; + + info!( + account, + fee = exact_fee, + entries = entries.len(), + inputs = selected_notes.len(), + total_input, + data_contract = %data_contract_id, + document_type = document_type_name, + "PrivateDocumentCreateFromShieldedPool" + ); + + // The journal records of a private contact request payment, keyed by + // these commitments, are discharged here only when the transition never + // left the wallet. + let journaled = private_contact.map(|_| { + entries + .iter() + .filter_map(|entry| { + entry + .as_slice() + .get(..32) + .and_then(|commitment| <[u8; 32]>::try_from(commitment).ok()) + }) + .collect::>() + }); + let built = async { + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; + let (state_transition, fee_used) = + build_private_document_create_from_shielded_pool_transition( + spends, + data_contract_id, + document_type_name.to_string(), + entries, + &change_addr, + &keys.full_viewing_key, + &keys.spend_auth_key, + anchor, + prover, + sdk.version(), + ) + .map_err(|e| PlatformWalletError::ShieldedBuildError(e.to_string()))?; + debug_assert_eq!( + fee_used, exact_fee, + "builder fee must match the reserved private document create fee" + ); + Ok::<_, PlatformWalletError>((state_transition, fee_used, anchor.to_bytes(), anchor_height)) + } + .await; + let (state_transition, fee_used, anchor_bytes, anchor_height) = match built { + Ok(built) => built, + Err(e) => { + // Nothing was saved or broadcast. + reservation.cancel().await; + return Err(e); + } + }; + // The spend's only outputs are change, so it records as an own spend + // with its fee and no counterparty. + let pending_entry = record_pending_activity( + store, + persister, + wallet_id, + id, + &views, + LiveEntryParams { + kind: ShieldedActivityKind::ShieldedSpend, + direction: ShieldedDirection::Out, + amount: 0, + fee: Some(fee_used), + counterparty: None, + memo: None, + actions: shielded_actions(&state_transition), + spent_notes: &selected_notes, + }, + ) + .await; + // Type 29 proves its spent nullifiers as a snapshot, which a competing + // spend of the same notes shares, as type 22's does: the affected-state + // wait accepts it (see `identity_top_up_from_pool_with_metadata`). + submit_private_document_payment( + PrivateDocumentSubmission { + store, + persister, + wallet_id, + id, + pending_entry, + selected_notes: &selected_notes, + reservation, + state_transition, + anchor: anchor_bytes, + anchor_height, + private_contact, + journaled: journaled.as_deref(), + payment_started, + }, + |state_transition| async move { + broadcast_shielded_spend_classified( + sdk, + &state_transition, + "private document create from shielded pool", + SpendResultWait::AffectedState, + ) + .await + }, + ) + .await +} + +/// A built private document create, and everything submitting it takes +/// ([`submit_private_document_payment`]). +struct PrivateDocumentSubmission<'a, S: ShieldedStore> { + store: &'a Arc>, + persister: Option<&'a WalletPersister>, + wallet_id: WalletId, + id: SubwalletId, + /// The spend's activity entry, recorded pending; it keys its records. + pending_entry: Option, + selected_notes: &'a [ShieldedNote], + reservation: NoteReservation, + state_transition: StateTransition, + anchor: [u8; 32], + /// The proved height of the recorded-anchor set `anchor` was chosen from. + anchor_height: u64, + private_contact: Option, + journaled: Option<&'a [[u8; 32]]>, + payment_started: &'a PaymentStarted, +} + +/// Record, save and broadcast a built private document create, then settle +/// its notes and saved record ([`settle_private_document_payment`]). +/// `broadcast` sends the transition and waits for its result: the production +/// flow passes the classified broadcast, a test intercepts it. +/// +/// Saved before the broadcast, not after an unconfirmed wait: a process that +/// exits once the node admitted the transition, before the wait returns, +/// still leaves the record a restart restores the request's reservation from, +/// and the redrive resolves. The payment is marked started once the record is +/// saved, with nothing awaited in between, and the notes are handed over only +/// then: a send dropped while the save waits for the store has saved and +/// broadcast nothing, and its guard releases the notes. A save that fails +/// stops the payment before the broadcast. +async fn submit_private_document_payment( + submission: PrivateDocumentSubmission<'_, S>, + broadcast: B, +) -> Result +where + S: ShieldedStore, + B: FnOnce(StateTransition) -> Fut, + Fut: Future>, +{ + let PrivateDocumentSubmission { + store, + persister, + wallet_id, + id, + pending_entry, + selected_notes, + mut reservation, + state_transition, + anchor, + anchor_height, + private_contact, + journaled, + payment_started, + } = submission; + let result = async { + let anchored = arm_pending_release( + store, + id, + anchor, + anchor_height, + &pending_entry, + selected_notes, + ) + .await?; + arm_redrive_before_broadcast( + store, + id, + pending_entry.as_ref().map(|entry| entry.id), + anchor, + selected_notes.iter().map(|note| note.nullifier).collect(), + &state_transition, + "private document create from shielded pool", + private_contact, + Some(payment_started), + anchor_height, + ) + .await?; + if anchored { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } + trace!("PrivateDocumentCreateFromShieldedPool: state transition saved, broadcasting..."); + broadcast(state_transition).await + } + .await; + + let activity_id = pending_entry.as_ref().map(|entry| entry.id); + settle_private_document_payment( + store, + persister, + wallet_id, + id, + &pending_entry, + activity_id, + reservation, + selected_notes, + journaled, + payment_started, + result.map(|(_proof, metadata)| metadata), + ) + .await +} + +/// Settle a private document payment's notes and saved record once its +/// broadcast and result wait returned `result`, as +/// [`private_document_create_from_pool_with_metadata`] does. +/// +/// A payment that executed marks its notes spent; one that may have executed +/// (unconfirmed) keeps them reserved and its saved record, which the redrive +/// pass resolves; one that did not execute releases them, and its record with +/// them. The private payment journal's records of a private contact request +/// payment (`journaled`, its entries' commitments) outlive all of that: they +/// are discharged here only when the transition never left the wallet +/// ([`SpendFailure::never_delivered`]), as nothing else can then execute it, +/// and `payment_started` tells the caller so +/// ([`PaymentStarted::note_never_delivered`]). Otherwise they stay until the +/// entry is read back. +/// +/// A transition that never left the wallet has its saved record, keyed by +/// `activity_id`, retired with the journal records in one durable write, +/// under the store lock the redrive pass claims each re-broadcast under +/// ([`NoteReservation::retire_undelivered`]). One whose record a redrive +/// pass claimed a re-broadcast of meanwhile, or that cannot be retired, may +/// leave the wallet after all: it is settled as unconfirmed, keeping its +/// notes, record and journal records. +#[allow(clippy::too_many_arguments)] +async fn settle_private_document_payment( + store: &Arc>, + persister: Option<&WalletPersister>, + wallet_id: WalletId, + id: SubwalletId, + pending_entry: &Option, + activity_id: Option<[u8; 32]>, + mut reservation: NoteReservation, + selected_notes: &[ShieldedNote], + journaled: Option<&[[u8; 32]]>, + payment_started: &PaymentStarted, + result: Result, +) -> Result { + match result { + Ok(metadata) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); + record_activity_status( + store, + persister, + wallet_id, + id, + pending_entry, + ShieldedActivityStatus::Confirmed, + None, + ) + .await; + if let Err(e) = finalize_pending(store, persister, wallet_id, id, selected_notes).await + { + warn!( + account = id.account_index, + error = %e, + "PrivateDocumentCreateFromShieldedPool broadcast succeeded but local \ + spent-state update failed; will heal on next sync" + ); + } + info!( + account = id.account_index, + "PrivateDocumentCreateFromShieldedPool broadcast succeeded" + ); + Ok(metadata) + } + Err(SpendFailure { + error: e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }, + .. + }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); Err(e) } + Err(SpendFailure { + error, + never_delivered: true, + }) => { + let commitments = journaled.unwrap_or_default(); + let retired = match activity_id { + Some(activity_id) => { + reservation + .retire_undelivered(&activity_id, commitments) + .await + } + None => { + reservation.hand_over(); + false + } + }; + if !retired { + // It may leave the wallet after all: everything stays, as for + // an unconfirmed payment. + return Err(PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: format!( + "its broadcast was refused before delivery ({error}), but its saved \ + transition was claimed for a re-broadcast or could not be retired" + ), + }); + } + record_activity_status( + store, + persister, + wallet_id, + id, + pending_entry, + ShieldedActivityStatus::Failed, + None, + ) + .await; + if journaled.is_some() { + if let Some(persister) = persister { + persister.note_discharged_private_entries(commitments.iter().copied()); + } + payment_started.note_never_delivered(); + } + Err(error) + } + Err(SpendFailure { error, .. }) => { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); + record_activity_status( + store, + persister, + wallet_id, + id, + pending_entry, + ShieldedActivityStatus::Failed, + None, + ) + .await; + reservation.cancel().await; + Err(error) + } } } @@ -1588,7 +2022,7 @@ pub async fn unshield( // reserve against `ShieldedFeeKind::Unshield` — reserving the base fee here would under-fund the // address-write cost and the builder would reject the spend (and the `fee_used == exact_fee` // debug assert below would fire). - let (selected_notes, total_input, exact_fee) = + let (selected_notes, total_input, exact_fee, mut reservation) = reserve_unspent_notes(sdk, store, id, amount, 2, ShieldedFeeKind::Unshield).await?; info!( @@ -1611,7 +2045,8 @@ pub async fn unshield( // it. A build failure leaves it `None` and records nothing. let mut pending_entry = None; let result = async { - let (spends, anchor) = extract_spends_and_anchor(sdk, store, &selected_notes).await?; + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; // Capture the recorded anchor before the builder consumes it, so a // broadcast-accepted-but-unconfirmed spend can be auto-released once // this anchor is pruned from Platform's recorded set. @@ -1660,7 +2095,19 @@ pub async fn unshield( }, ) .await; - arm_pending_release(store, id, anchor_bytes, &pending_entry, &selected_notes).await; + if arm_pending_release( + store, + id, + anchor_bytes, + anchor_height, + &pending_entry, + &selected_notes, + ) + .await? + { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } trace!("Unshield: state transition built, broadcasting..."); broadcast_shielded_spend_with_redrive( @@ -1669,6 +2116,7 @@ pub async fn unshield( id, &pending_entry, anchor_bytes, + anchor_height, &selected_notes, &state_transition, "unshield", @@ -1680,6 +2128,8 @@ pub async fn unshield( match result { Ok(_) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); record_activity_status( store, persister, @@ -1730,8 +2180,15 @@ pub async fn unshield( // the outcome is genuinely unknown here — a later scan that finds // the spend will flip the row to Confirmed (its id matches), and // until then surfacing it as Pending is honest. - Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => Err(e), + Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); + Err(e) + } Err(e) => { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); // Definitive failure: the spend never executed. Mark the // activity row Failed (upsert by id) before releasing notes. record_activity_status( @@ -1744,7 +2201,7 @@ pub async fn unshield( None, ) .await; - cancel_pending(store, id, &selected_notes).await; + reservation.cancel().await; Err(e) } } @@ -1776,7 +2233,7 @@ pub async fn transfer( // ShieldedTransfer is carved with the base `compute_minimum_shielded_fee`, so reserve // against `ShieldedFeeKind::Base`. - let (selected_notes, total_input, exact_fee) = + let (selected_notes, total_input, exact_fee, mut reservation) = reserve_unspent_notes(sdk, store, id, amount, 2, ShieldedFeeKind::Base).await?; info!( @@ -1790,7 +2247,8 @@ pub async fn transfer( let mut pending_entry = None; let result = async { - let (spends, anchor) = extract_spends_and_anchor(sdk, store, &selected_notes).await?; + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; // Capture the recorded anchor before the builder consumes it, so a // broadcast-accepted-but-unconfirmed spend can be auto-released once // this anchor is pruned from Platform's recorded set. @@ -1839,7 +2297,19 @@ pub async fn transfer( }, ) .await; - arm_pending_release(store, id, anchor_bytes, &pending_entry, &selected_notes).await; + if arm_pending_release( + store, + id, + anchor_bytes, + anchor_height, + &pending_entry, + &selected_notes, + ) + .await? + { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } trace!("Shielded transfer: state transition built, broadcasting..."); broadcast_shielded_spend_with_redrive( @@ -1848,6 +2318,7 @@ pub async fn transfer( id, &pending_entry, anchor_bytes, + anchor_height, &selected_notes, &state_transition, "transfer", @@ -1859,6 +2330,8 @@ pub async fn transfer( match result { Ok(_) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); record_activity_status( store, persister, @@ -1889,8 +2362,15 @@ pub async fn transfer( // Ambiguous post-broadcast confirmation failure: leave the // reservation (and the Pending activity row) in place — a later // scan flips it to Confirmed (see `unshield`'s outer match). - Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => Err(e), + Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); + Err(e) + } Err(e) => { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); record_activity_status( store, persister, @@ -1901,7 +2381,7 @@ pub async fn transfer( None, ) .await; - cancel_pending(store, id, &selected_notes).await; + reservation.cancel().await; Err(e) } } @@ -1938,7 +2418,7 @@ pub async fn withdraw( // `ShieldedFeeKind::Withdrawal` — reserving the base fee here would under-fund the document // cost and the builder would reject the spend (and the `fee_used == exact_fee` debug assert // below would fire). - let (selected_notes, total_input, exact_fee) = + let (selected_notes, total_input, exact_fee, mut reservation) = reserve_unspent_notes(sdk, store, id, amount, 2, ShieldedFeeKind::Withdrawal).await?; info!( @@ -1957,7 +2437,8 @@ pub async fn withdraw( let mut pending_entry = None; let result = async { - let (spends, anchor) = extract_spends_and_anchor(sdk, store, &selected_notes).await?; + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; // Capture the recorded anchor before the builder consumes it, so a // broadcast-accepted-but-unconfirmed spend can be auto-released once // this anchor is pruned from Platform's recorded set. @@ -2008,7 +2489,19 @@ pub async fn withdraw( }, ) .await; - arm_pending_release(store, id, anchor_bytes, &pending_entry, &selected_notes).await; + if arm_pending_release( + store, + id, + anchor_bytes, + anchor_height, + &pending_entry, + &selected_notes, + ) + .await? + { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } trace!("Shielded withdrawal: state transition built, broadcasting..."); broadcast_shielded_spend_with_redrive( @@ -2017,6 +2510,7 @@ pub async fn withdraw( id, &pending_entry, anchor_bytes, + anchor_height, &selected_notes, &state_transition, "withdraw", @@ -2028,6 +2522,8 @@ pub async fn withdraw( match result { Ok(_) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); record_activity_status( store, persister, @@ -2058,8 +2554,15 @@ pub async fn withdraw( // Ambiguous post-broadcast confirmation failure: leave the // reservation (and the Pending activity row) in place — a later // scan flips it to Confirmed (see `unshield`'s outer match). - Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => Err(e), + Err(e @ PlatformWalletError::ShieldedSpendUnconfirmed { .. }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); + Err(e) + } Err(e) => { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); record_activity_status( store, persister, @@ -2070,7 +2573,7 @@ pub async fn withdraw( None, ) .await; - cancel_pending(store, id, &selected_notes).await; + reservation.cancel().await; Err(e) } } @@ -2131,7 +2634,7 @@ where // — the fee is metered FROM the denomination at execution). The reservation also gates on // `denomination > predicted_fee` so the new identity can't be created with a non-positive // balance. Orchard's BundleType::DEFAULT pads single-spend bundles to a 2-action floor. - let (selected_notes, total_input, predicted_fee) = + let (selected_notes, total_input, predicted_fee, mut reservation) = reserve_unspent_notes_for_denomination(sdk, store, id, denomination, 2, num_keys).await?; info!( @@ -2163,7 +2666,8 @@ where // outer match; it lives here so the flip can see it. let mut pending_entry = None; let result = async { - let (spends, anchor) = extract_spends_and_anchor(sdk, store, &selected_notes).await?; + let (spends, anchor, anchor_height) = + extract_spends_and_recorded_anchor(sdk, store, &selected_notes).await?; // Capture the recorded anchor before the builder consumes it, so a // broadcast-accepted-but-unconfirmed create can be auto-released once // this anchor is pruned from Platform's recorded set. @@ -2214,7 +2718,18 @@ where }, ) .await; - arm_pending_release(store, id, anchor_bytes, &pending_entry, &selected_notes).await; + if arm_pending_release( + store, + id, + anchor_bytes, + anchor_height, + &pending_entry, + &selected_notes, + ) + .await? { + // Each reservation carries its anchor: a pruned anchor releases it. + reservation.hand_over(); + } trace!("IdentityCreateFromShieldedPool: built, broadcasting via SDK..."); // Stage the broadcast and the result-wait SEPARATELY (instead of one `broadcast_and_wait`) @@ -2236,42 +2751,36 @@ where ) .map_err(|e| PlatformWalletError::ShieldedBuildError(e.to_string()))?; - // Broadcast (relay-ACK only). Definitive failures — a consensus-verdict CheckTx - // rejection, or a transport failure that proves nothing was delivered (see - // `broadcast_definitely_failed`; connect-refused/offline is the common case) — map to - // `ShieldedBroadcastFailed` and let the outer match release the reservation, exactly as - // before. No-verdict failures (`AlreadyExists` proves the transition IS already in the - // mempool or on chain after a lost-ACK retry; timeouts merely allow it) fall through to - // the result wait below, whose ambiguous arm already owns the fetch-by-derived-id - // fallback — so the in-flight tx gets confirmed (or held as unconfirmed) instead of - // being reported as failed. - match st.broadcast(sdk, None).await { - Ok(()) => {} - Err(e) if broadcast_definitely_failed(&e) => { + // Broadcast (relay-ACK only). Only a transition that never left the wallet + // (`BroadcastDelivery::NeverDelivered`: refused before anything was sent) maps to + // `ShieldedBroadcastFailed`, + // which lets the outer match release the reservation. Every other failure may have + // delivered it and falls through to the result wait below, whose ambiguous arm owns the + // fetch-by-derived-id fallback — so the in-flight tx gets confirmed (or held as + // unconfirmed) instead of being reported as failed. + match broadcast_tracking_delivery(sdk, &st).await { + BroadcastDelivery::Admitted => {} + BroadcastDelivery::NeverDelivered(e) => { return Err(PlatformWalletError::ShieldedBroadcastFailed(e.to_string())); } - Err(e) => { + BroadcastDelivery::MaybeDelivered(e) => { warn!( derived_id = %identity_id, error = %e, - "IdentityCreateFromShieldedPool: broadcast returned no verdict; the \ - transition may have been admitted — falling through to the result wait" + "IdentityCreateFromShieldedPool: broadcast may have been delivered — \ + falling through to the result wait" ); } } - // Wait for proven execution. Classify the failure: - // - `StateTransitionBroadcastError` WITH a consensus `cause` = Platform DEFINITIVELY - // reported the transition's own execution error (it ran and was rejected on its - // merits; the serialized consensus error is the verdict). The identity does not - // exist; keep today's behavior (release the reservation via - // `ShieldedBroadcastFailed`). - // - any other error (DriveProofError / Proof / InvalidProvedResponse / TimeoutReached / - // DapiClientError / …) = AMBIGUOUS: the broadcast was accepted and the transition may - // have executed even though we couldn't fetch/verify its result proof. That includes - // cause-less `StateTransitionBroadcastError`s — DAPI encodes its own wait-side failures - // (timeouts, internal errors) that way, with empty consensus data. Fall back to - // fetching the identity by its pre-derived id before deciding it doesn't exist. + // Wait for proven execution. Every failure is AMBIGUOUS: the broadcast reached an + // endpoint and the transition may have executed even though we couldn't fetch or + // verify its result proof. That includes a `StateTransitionBroadcastError` with a + // consensus `cause`: the endpoint that reports it is unauthenticated, no proof or quorum + // signature backs it, and one that kept or forwarded the transition can answer with any + // verdict. It also includes cause-less ones, as DAPI encodes its own wait-side failures + // (timeouts, internal errors) that way. Fall back to fetching the identity by its + // pre-derived id before deciding it is unconfirmed. // An identity-create-from-shielded-pool proof authenticates the spent // nullifiers and resulting identity as a snapshot; it cannot bind the // complete Orchard request, so accept the affected-state outcome. @@ -2280,9 +2789,6 @@ where .await { Ok(result) => result, - Err(dash_sdk::Error::StateTransitionBroadcastError(e)) if e.cause.is_some() => { - return Err(PlatformWalletError::ShieldedBroadcastFailed(e.to_string())); - } Err(wait_err) => { warn!( derived_id = %identity_id, @@ -2324,6 +2830,7 @@ where id, &pending_entry, anchor_bytes, + anchor_height, &selected_notes, &st, "identity_create", @@ -2390,6 +2897,8 @@ where match result { Ok((identity_id, identity)) => { + // It executed: the outcome owns the notes, which it marks spent. + reservation.hand_over(); record_activity_status( store, persister, @@ -2422,7 +2931,7 @@ where Ok((identity_id, identity)) } // The broadcast was accepted but its result couldn't be confirmed and the identity wasn't - // found by a direct fetch. Do NOT `cancel_pending` here: `pending_nullifiers` is in-memory + // found by a direct fetch. Do NOT release the notes here: `pending_nullifiers` is in-memory // only (see `SubwalletState`, "never persisted; the next sync after a crash reconciles"), // and `mark_spent` during nullifier sync clears matching reservations. So if the transition // actually executed, the next sync promotes these notes to spent; if it truly never landed, @@ -2430,9 +2939,16 @@ where // invite double-spend attempts against notes that may already be consumed on chain — the // very hazard this variant exists to prevent. The activity row likewise stays `Pending` // (a later scan that finds the identity's change note flips it to Confirmed). - Err(e @ PlatformWalletError::ShieldedBroadcastUnconfirmed { .. }) => Err(e), + Err(e @ PlatformWalletError::ShieldedBroadcastUnconfirmed { .. }) => { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); + Err(e) + } Err(e) => { if error_releases_note_reservation(&e) { + // It did not execute: the guard owns the notes until they are + // released, so a drop on the way still releases them. + reservation.reclaim(); // Definitive failure: the identity was never created. Mark // the activity row Failed (upsert by id) before releasing. record_activity_status( @@ -2445,7 +2961,10 @@ where None, ) .await; - cancel_pending(store, id, &selected_notes).await; + reservation.cancel().await; + } else { + // It may have executed: the reservation stays, released by sync. + reservation.hand_over(); } Err(e) } @@ -2558,11 +3077,15 @@ const MAX_ANCHOR_PROBE_DEPTH: usize = 100; /// recorded root it returns the retryable /// [`PlatformWalletError::ShieldedNoRecordedAnchor`] rather than broadcasting a /// spend Platform is guaranteed to reject. -async fn extract_spends_and_anchor( +/// +/// Returns the proved height of the recorded anchor set the anchor was chosen +/// from too: the anchor was in the set at that height, so a set proved at or +/// above it that lacks the anchor shows it pruned rather than newer. +async fn extract_spends_and_recorded_anchor( sdk: &Arc, store: &Arc>, notes: &[ShieldedNote], -) -> Result<(Vec, Anchor), PlatformWalletError> { +) -> Result<(Vec, Anchor, u64), PlatformWalletError> { // Nothing selected — fail before the network round-trip. if notes.is_empty() { return Err(PlatformWalletError::ShieldedBuildError( @@ -2573,8 +3096,8 @@ async fn extract_spends_and_anchor( // Fetch the recorded anchor set OUTSIDE the store lock so the network // round-trip doesn't serialize with other store users, and so the lock is // held only for the mutually-consistent depth/witness probe below. - let dash_sdk::query_types::ShieldedAnchors(recorded_anchors) = - dash_sdk::query_types::ShieldedAnchors::fetch_current(sdk).await?; + let (dash_sdk::query_types::ShieldedAnchors(recorded_anchors), metadata) = + dash_sdk::query_types::ShieldedAnchors::fetch_current_with_metadata(sdk).await?; let recorded: HashSet<[u8; 32]> = recorded_anchors.into_iter().collect(); // Hold a single read lock across the whole probe so the checkpoint depths @@ -2583,6 +3106,7 @@ async fn extract_spends_and_anchor( // under us. let store = store.read().await; select_recorded_spends(&*store, notes, &recorded) + .map(|(spends, anchor)| (spends, anchor, metadata.height)) } /// Pick the shallowest checkpoint depth whose tree root is in `recorded`, @@ -2784,8 +3308,10 @@ async fn mark_notes_spent( /// The reservation is in-memory only — see /// [`ShieldedStore::mark_pending`] for the crash-recovery note. /// Callers must pair this with [`finalize_pending`] (on -/// broadcast success) or [`cancel_pending`] (on failure) so the -/// reservation is always released. +/// broadcast success) or [`NoteReservation::cancel`] (on failure) so the +/// reservation is always released; until they hand it over +/// ([`NoteReservation::hand_over`]), the returned guard releases it when +/// the spend is dropped. async fn reserve_unspent_notes( sdk: &Arc, store: &Arc>, @@ -2793,59 +3319,225 @@ async fn reserve_unspent_notes( amount: u64, outputs: usize, fee_kind: ShieldedFeeKind, -) -> Result<(Vec, u64, u64), PlatformWalletError> { +) -> Result<(Vec, u64, u64, NoteReservation), PlatformWalletError> { // All amount-based spends reject zero before waiting on a concurrent sync's store lock. ensure_positive_amount(amount)?; + reserve_notes_covering(sdk, store, id, amount, outputs, fee_kind).await +} - let mut store = store.write().await; - let unspent = store +/// Select and reserve, in one write-locked critical section, notes covering +/// `amount` plus the fee of `fee_kind`, which [`reserve_unspent_notes`] does +/// for a positive amount. A spend paying only its fee (an +/// `PrivateDocumentCreateFromShieldedPool`) reserves with an `amount` of 0. +/// Callers must pair this with [`finalize_pending`] / [`NoteReservation::cancel`], and +/// hand the guard over once they own the reservation. +async fn reserve_notes_covering( + sdk: &Arc, + store: &Arc>, + id: SubwalletId, + amount: u64, + outputs: usize, + fee_kind: ShieldedFeeKind, +) -> Result<(Vec, u64, u64, NoteReservation), PlatformWalletError> { + let mut guard = store.write().await; + let unspent = guard .get_unspent_notes(id) .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; let (selected, total_input, exact_fee) = select_notes_with_fee(&unspent, amount, outputs, fee_kind, sdk.version())?.into_owned(); - for note in &selected { - store - .mark_pending(id, ¬e.nullifier) - .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; - } - Ok((selected, total_input, exact_fee)) + let reservation = mark_notes_pending(&mut *guard, store, id, &selected)?; + Ok((selected, total_input, exact_fee, reservation)) } -/// Exact-equality sibling of [`reserve_unspent_notes`] for -/// `IdentityCreateFromShieldedPool`: select + reserve notes covering exactly `denomination` -/// (the fee is metered FROM the denomination, not added to the target) in one write-locked -/// critical section, gating on `denomination > predicted_fee` via -/// [`select_notes_for_denomination`]. Returns the selected notes, total input value, and the -/// predicted fee. Callers must pair this with [`finalize_pending`] / [`cancel_pending`]. -async fn reserve_unspent_notes_for_denomination( - sdk: &Arc, +/// Mark `notes` pending under the held store lock, returning the guard that +/// releases them unless handed over. A failure part way releases those +/// already marked. +fn mark_notes_pending( + locked: &mut S, store: &Arc>, id: SubwalletId, - denomination: u64, - min_actions: usize, - num_keys: usize, -) -> Result<(Vec, u64, u64), PlatformWalletError> { - // Reject zero or any other non-member before waiting on a concurrent sync's store lock. - ensure_allowed_denomination(denomination, sdk.version())?; + notes: &[ShieldedNote], +) -> Result, PlatformWalletError> { + let reservation = next_note_reservation(); + let mut marked = Vec::with_capacity(notes.len()); + for note in notes { + if let Err(e) = locked.mark_pending_by(id, ¬e.nullifier, reservation) { + release_reserved_notes(locked, id, &marked, reservation); + return Err(PlatformWalletError::ShieldedStoreError(e.to_string())); + } + marked.push(note.nullifier); + } + Ok(NoteReservation { + store: Arc::clone(store), + id, + nullifiers: marked, + reservation, + handed_over: false, + }) +} - let mut store = store.write().await; - let unspent = store - .get_unspent_notes(id) - .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; - let (selected, total_input, predicted_fee) = select_notes_for_denomination( - &unspent, - denomination, - min_actions, - num_keys, +/// A spend attempt's reservation id, unique in the process: what its note +/// reservations are owned by ([`ShieldedStore::mark_pending_by`]). +fn next_note_reservation() -> u64 { + static NEXT: AtomicU64 = AtomicU64::new(1); + NEXT.fetch_add(1, AtomicOrdering::Relaxed) +} + +/// Notes one spend reserved ([`ShieldedStore::mark_pending`]), released when +/// dropped before the spend hands them over ([`Self::hand_over`]): a spend +/// future dropped before its notes have an owner that frees them (an anchor +/// on each reservation, which a pruned anchor releases, or the outcome +/// handling that finalizes or cancels them) leaves them spendable again, as +/// [`NoteReservation::cancel`] would, rather than excluded for the rest of the +/// session. The release takes the store lock when it is free, and otherwise +/// runs on a spawned task once it frees; outside a runtime the notes stay +/// reserved until restart. +pub(super) struct NoteReservation { + store: Arc>, + id: SubwalletId, + nullifiers: Vec<[u8; 32]>, + /// The attempt that owns the reservations: a release clears a note + /// only while it still holds it, so a cleanup that runs after a purge, + /// a rebind or a wallet's replacement never releases a newer attempt's + /// notes or deletes its records. + reservation: u64, + handed_over: bool, +} + +impl NoteReservation { + /// The spend owns the reservation from now on. + pub(super) fn hand_over(&mut self) { + self.handed_over = true; + } + + /// Own the reservation again, for a spend now known not to have executed: dropped from + /// here on, the guard releases the notes, whatever owned them before. + pub(super) fn reclaim(&mut self) { + self.handed_over = false; + } + + /// For a spend whose transition never left the wallet: retire its saved + /// record `activity_id` with the journal records of `commitments` in one + /// durable write ([`ShieldedStore::retire_undelivered_spend`]), and + /// release the notes under the same store guard. Returns whether it was + /// retired. + /// + /// When it was not (a redrive pass claimed a re-broadcast of the record, + /// the record is gone, or the write failed), nothing changed and the + /// notes stay reserved, handed over to the spend's resolution. So do they + /// when the call is dropped while it waits for the store: the record may + /// still be saved, and a note released under a saved transition could be + /// spent twice. + pub(super) async fn retire_undelivered( + mut self, + activity_id: &[u8; 32], + commitments: &[[u8; 32]], + ) -> bool { + self.handed_over = true; + let mut store = self.store.write().await; + match store.retire_undelivered_spend(self.id, activity_id, commitments) { + Ok(true) => { + release_reserved_notes(&mut *store, self.id, &self.nullifiers, self.reservation); + self.nullifiers.clear(); + true + } + Ok(false) => false, + Err(e) => { + warn!( + error = %e, + "a never-delivered spend's record was not retired; it is kept with its notes" + ); + false + } + } + } + + /// Release the notes now, for a spend that did not execute. + /// The guard owns them until the release is done: a spend dropped while it waits for the + /// store lock still releases them on drop. + pub(super) async fn cancel(mut self) { + self.handed_over = false; + let mut store = self.store.write().await; + release_reserved_notes(&mut *store, self.id, &self.nullifiers, self.reservation); + self.nullifiers.clear(); + } +} + +impl Drop for NoteReservation { + fn drop(&mut self) { + if self.handed_over || self.nullifiers.is_empty() { + return; + } + let (id, reservation) = (self.id, self.reservation); + let nullifiers = std::mem::take(&mut self.nullifiers); + if let Ok(mut store) = self.store.try_write() { + release_reserved_notes(&mut *store, id, &nullifiers, reservation); + return; + } + let Ok(runtime) = tokio::runtime::Handle::try_current() else { + warn!( + "a dropped spend's notes were reserved outside a runtime; they stay until restart" + ); + return; + }; + let store = Arc::clone(&self.store); + runtime.spawn(async move { + let mut store = store.write().await; + release_reserved_notes(&mut *store, id, &nullifiers, reservation); + }); + } +} + +/// Clear the reservations of `nullifiers` the spend attempt `reservation` +/// still holds ([`ShieldedStore::clear_pending_reserved_by`]). +fn release_reserved_notes( + store: &mut S, + id: SubwalletId, + nullifiers: &[[u8; 32]], + reservation: u64, +) { + for nullifier in nullifiers { + if let Err(e) = store.clear_pending_reserved_by(id, nullifier, reservation) { + warn!( + error = %e, + "releasing a dropped spend's note failed; the next note scan will reconcile" + ); + } + } +} + +/// Exact-equality sibling of [`reserve_unspent_notes`] for +/// `IdentityCreateFromShieldedPool`: select + reserve notes covering exactly `denomination` +/// (the fee is metered FROM the denomination, not added to the target) in one write-locked +/// critical section, gating on `denomination > predicted_fee` via +/// [`select_notes_for_denomination`]. Returns the selected notes, total input value, and the +/// predicted fee. Callers must pair this with [`finalize_pending`] / [`NoteReservation::cancel`], and +/// hand the guard over once they own the reservation. +async fn reserve_unspent_notes_for_denomination( + sdk: &Arc, + store: &Arc>, + id: SubwalletId, + denomination: u64, + min_actions: usize, + num_keys: usize, +) -> Result<(Vec, u64, u64, NoteReservation), PlatformWalletError> { + // Reject zero or any other non-member before waiting on a concurrent sync's store lock. + ensure_allowed_denomination(denomination, sdk.version())?; + + let mut guard = store.write().await; + let unspent = guard + .get_unspent_notes(id) + .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; + let (selected, total_input, predicted_fee) = select_notes_for_denomination( + &unspent, + denomination, + min_actions, + num_keys, sdk.version(), )? .into_owned(); - for note in &selected { - store - .mark_pending(id, ¬e.nullifier) - .map_err(|e| PlatformWalletError::ShieldedStoreError(e.to_string()))?; - } - Ok((selected, total_input, predicted_fee)) + let reservation = mark_notes_pending(&mut *guard, store, id, &selected)?; + Ok((selected, total_input, predicted_fee, reservation)) } /// Promote a successful broadcast: mark the notes spent (which @@ -2862,25 +3554,6 @@ async fn finalize_pending( mark_notes_spent(store, persister, wallet_id, id, notes).await } -/// Roll back a reservation when the broadcast / wait fails. -/// Best-effort and doesn't surface its own errors — the caller -/// is already returning the broadcast error. -async fn cancel_pending( - store: &Arc>, - id: SubwalletId, - notes: &[ShieldedNote], -) { - let mut store = store.write().await; - for note in notes { - if let Err(e) = store.clear_pending(id, ¬e.nullifier) { - tracing::warn!( - error = %e, - "cancel_pending: clear_pending failed; the next note scan will reconcile" - ); - } - } -} - /// Record the recorded `anchor` the spend was built against and the /// linked activity entry on every selected note's reservation, so a /// spend that ends broadcast-accepted-but-unconfirmed can be released @@ -2889,32 +3562,40 @@ async fn cancel_pending( /// /// No-op when no activity entry was recorded (an output-less bundle — /// unreachable for our own builders, which always carry a visible -/// output). Best-effort: a store write failure only means the -/// reservation won't self-release on a pruned anchor (it still frees -/// on the next restart), so it must never abort a spend about to -/// broadcast. A success (`finalize_pending`) or a definite failure -/// (`cancel_pending`) removes the entry, so only an ambiguous +/// output). A success (`finalize_pending`) or a definite failure +/// (`NoteReservation::cancel`) removes the entry, so only an ambiguous /// unconfirmed outcome leaves it carrying the anchor. +/// +/// Returns whether every reservation now carries the anchor, with +/// `anchor_height`, the proved height of the anchor set it was read from, so +/// an anchor proven pruned releases each. A store write that fails stops the spend before it +/// is broadcast ([`PlatformWalletError::ShieldedStoreError`]): the caller's +/// reservation guard still owns the notes, so they are released, rather +/// than a spend that may execute leaving reservations no pruned anchor +/// frees. async fn arm_pending_release( store: &Arc>, id: SubwalletId, anchor: [u8; 32], + anchor_height: u64, pending_entry: &Option, notes: &[ShieldedNote], -) { +) -> Result { let Some(entry) = pending_entry else { - return; + return Ok(false); }; let mut store = store.write().await; for note in notes { - if let Err(e) = store.set_pending_spend(id, ¬e.nullifier, anchor, entry.id) { - warn!( - error = %e, - "set_pending_spend failed; this reservation won't self-release on a pruned \ - anchor (it still frees on the next restart)" - ); - } + store + .set_pending_spend(id, ¬e.nullifier, anchor, entry.id, Some(anchor_height)) + .map_err(|e| { + PlatformWalletError::ShieldedStoreError(format!( + "the spend's note reservation could not carry its anchor, so it is not \ + broadcast: {e}" + )) + })?; } + Ok(true) } /// Maximum sync-time re-broadcast attempts for a @@ -2937,6 +3618,7 @@ async fn broadcast_shielded_spend_with_redrive( id: SubwalletId, pending_entry: &Option, anchor: [u8; 32], + anchor_height: u64, notes: &[ShieldedNote], state_transition: &StateTransition, operation: &'static str, @@ -2948,6 +3630,7 @@ async fn broadcast_shielded_spend_with_redrive( id, pending_entry, anchor, + anchor_height, notes, state_transition, operation, @@ -2964,6 +3647,7 @@ async fn broadcast_shielded_spend_with_redrive_with_metadata( id: SubwalletId, pending_entry: &Option, anchor: [u8; 32], + anchor_height: u64, notes: &[ShieldedNote], state_transition: &StateTransition, operation: &'static str, @@ -2979,6 +3663,7 @@ async fn broadcast_shielded_spend_with_redrive_with_metadata( id, pending_entry, anchor, + anchor_height, notes, state_transition, operation, @@ -2988,15 +3673,163 @@ async fn broadcast_shielded_spend_with_redrive_with_metadata( result } +/// Save the re-drivable record of a spend durably before it is broadcast, +/// as identity-funded shields arm theirs: an exit after the node admitted +/// the transition, before its result returns, leaves the record. The record +/// is keyed by the spend's `activity_id` and carries its `nullifiers`, and +/// the private contact request the spend pays for when it does. A spend +/// paying for one is a private document create (type 29), and the private +/// payment journal keeps a record of each entry it appends, read from the +/// transition, in the same durable write ([`ShieldedStore::arm_private_contact_payment`]). +/// A spend whose record cannot be saved, or whose entries do not read, is +/// refused before anything is broadcast, as the unknown outcome could then +/// not be recovered. `payment_started`, when given, is marked right after +/// the record is saved, under the same store lock: dropped while it waits +/// for the lock, the spend leaves neither a record nor a mark. +/// `anchor_recorded_at` is the proved height of the recorded-anchor set the +/// spend's anchor was chosen from, kept in the journal record: an expiry +/// proof needs the anchor absent from a set proved at or above it. +#[allow(clippy::too_many_arguments)] +pub(crate) async fn arm_redrive_before_broadcast( + store: &Arc>, + id: SubwalletId, + activity_id: Option<[u8; 32]>, + anchor: [u8; 32], + nullifiers: Vec<[u8; 32]>, + state_transition: &StateTransition, + operation: &'static str, + private_contact: Option, + payment_started: Option<&PaymentStarted>, + anchor_recorded_at: u64, +) -> Result<(), PlatformWalletError> { + use dpp::serialization::PlatformSerializable; + + let activity_id = activity_id.ok_or_else(|| { + PlatformWalletError::ShieldedBuildError(format!( + "{operation}: no activity entry keys the record of this spend, so it is not \ + broadcast" + )) + })?; + let st_bytes = state_transition.serialize_to_bytes().map_err(|e| { + PlatformWalletError::ShieldedBuildError(format!( + "{operation}: the transition does not serialize for its record, so it is not \ + broadcast: {e}" + )) + })?; + let journal = match private_contact { + Some(payment) => Some(( + payment, + private_document_commitments(state_transition).ok_or_else(|| { + PlatformWalletError::ShieldedBuildError(format!( + "{operation}: the entries this private contact request payment appends do \ + not read, so it is not broadcast" + )) + })?, + )), + None => None, + }; + let redrive = PendingRedrive { + activity_id, + anchor, + nullifiers, + st_bytes, + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: Some(anchor_recorded_at), + }; + let mut store = store.write().await; + match journal { + Some((payment, commitments)) => store.arm_private_contact_payment( + id, + redrive, + payment, + &commitments, + anchor_recorded_at, + ), + None => store.arm_redrive(id, redrive), + } + .map_err(|e| { + PlatformWalletError::ShieldedStoreError(format!( + "{operation}: the record of this spend could not be saved, so it is not broadcast: \ + {e}" + )) + })?; + if let Some(payment_started) = payment_started { + payment_started.mark(); + } + Ok(()) +} + +/// Discharge `wallet_id`'s journal records of the private contact request +/// entries a private sync read back and recorded durably, as `persister` +/// noted them (`WalletPersister::note_read_back_private_entries`). One whose +/// discharge fails is noted again for the next call. +pub(crate) fn discharge_read_back_private_payments( + store: &mut S, + persister: &WalletPersister, + wallet_id: WalletId, +) { + let mut kept = Vec::new(); + let mut discharged = Vec::new(); + for commitment in persister.take_read_back_private_entries() { + match store.discharge_private_payment(wallet_id, &commitment) { + Ok(_) => discharged.push(commitment), + Err(e) => { + warn!( + error = %e, + "a read-back private payment's journal record was not discharged; retried \ + later" + ); + kept.push(commitment); + } + } + } + persister.note_discharged_private_entries(discharged); + if !kept.is_empty() { + persister.note_read_back_private_entries(kept); + } +} + +/// The commitments of the entries the private document create (type 29) +/// `state_transition` appends, each entry's first 32 bytes; `None` for +/// another transition, one appending no entry, or an entry shorter than its +/// commitment. +pub(crate) fn private_document_commitments( + state_transition: &StateTransition, +) -> Option> { + use dpp::state_transition::private_document_create_from_shielded_pool_transition::accessors::PrivateDocumentCreateFromShieldedPoolTransitionAccessorsV0; + + let StateTransition::PrivateDocumentCreateFromShieldedPool(transition) = state_transition + else { + return None; + }; + if transition.entries().is_empty() { + return None; + } + transition + .entries() + .iter() + .map(|entry| { + entry + .as_slice() + .get(..32) + .and_then(|commitment| <[u8; 32]>::try_from(commitment).ok()) + }) + .collect() +} + /// Persist the re-drivable record for an ambiguous spend. Best-effort: /// a failure here only demotes the resolution path to the anchor-prune /// backstop (plus restart-loss of the reservation), never fails the /// spend call itself — the ambiguity already happened. +#[allow(clippy::too_many_arguments)] async fn arm_redrive_record( store: &Arc>, id: SubwalletId, pending_entry: &Option, anchor: [u8; 32], + anchor_height: u64, notes: &[ShieldedNote], state_transition: &StateTransition, operation: &'static str, @@ -3026,6 +3859,7 @@ async fn arm_redrive_record( attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: Some(anchor_height), }; if let Err(e) = store.write().await.arm_redrive(id, redrive) { warn!( @@ -3071,9 +3905,9 @@ enum RedriveBroadcastOutcome { /// `NullifierAlreadySpent`: the ORIGINAL broadcast executed — a /// success signal, never a failure. AlreadyExecuted, - /// Any other consensus verdict: the transition can never execute. - DefinitiveRejection, - /// Transport noise / `AlreadyExists` / anything non-definitive. + /// Any other outcome, a consensus verdict included: the original's + /// broadcast was ambiguous, so a refusal of the re-broadcast does not + /// prove it can never execute. Inconclusive, } @@ -3081,7 +3915,6 @@ fn classify_redrive_broadcast(result: &Result<(), dash_sdk::Error>) -> RedriveBr match result { Ok(()) => RedriveBroadcastOutcome::Accepted, Err(e) if is_nullifier_already_spent(e) => RedriveBroadcastOutcome::AlreadyExecuted, - Err(e) if carries_consensus_rejection(e) => RedriveBroadcastOutcome::DefinitiveRejection, Err(_) => RedriveBroadcastOutcome::Inconclusive, } } @@ -3109,6 +3942,45 @@ async fn bump_redrive_attempts_logged( } } +/// Claim one re-broadcast of `redrive`, durably, from the record as it is now, +/// under the store write lock a never-delivered spend's retirement takes +/// ([`ShieldedStore::retire_undelivered_spend`]): the attempt is counted +/// before the broadcast starts, so a retirement after the claim sees it and +/// refuses. A record retired, armed again with other bytes, or out of attempts +/// since `redrive` was read is not claimed, and not broadcast. Returns the +/// attempt count, or `None` when nothing was claimed: a counter that fails to +/// persist claims nothing. +async fn claim_redrive_attempt( + store: &Arc>, + id: SubwalletId, + redrive: &PendingRedrive, +) -> Option { + let mut store = store.write().await; + let live = store.pending_redrives(id).is_ok_and(|current| { + current.iter().any(|record| { + record.activity_id == redrive.activity_id + && record.st_bytes == redrive.st_bytes + && record.attempts < MAX_REDRIVE_ATTEMPTS + }) + }); + if !live { + debug!("redrive: the record was retired or changed since it was read; not re-broadcast"); + return None; + } + match store.bump_redrive_attempts(id, &redrive.activity_id) { + Ok(attempts) if attempts > 0 => Some(attempts), + Ok(_) => None, + Err(e) => { + warn!( + error = %e, + "redrive: the attempt could not be counted, so it is not re-broadcast; the next \ + pass retries" + ); + None + } + } +} + /// Re-broadcast every unresolved ShieldFromIdentity transition for `wallet_id`. /// These records have no nullifiers or spending anchor, so they must not depend /// on the note-spend stranded-release pass. Exact-byte retries remain safe even @@ -3232,11 +4104,16 @@ pub(super) async fn redrive_pending_identity_shields( /// - accepted / inconclusive → count the attempt; wait for the next scan; /// - `NullifierAlreadySpent` → the original executed; the next scan /// confirms — touch nothing; -/// - any other consensus verdict → provably dead NOW: release the -/// reservation and flip the activity row to Failed, hours before the -/// prune backstop would; +/// - any other consensus verdict → inconclusive too: a node refusing the +/// re-broadcast does not show that no other node holds the original, +/// whose broadcast was ambiguous, so the notes stay reserved; /// - pruned anchor / exhausted attempts → leave it to the -/// prune-backstop release pass. +/// prune-backstop release pass, which releases the notes once the +/// anchor is pruned and the spend can never execute. +/// +/// A record whose transition does not decode is kept, never dropped: it +/// may be the only evidence of a payment that can still execute. Its notes +/// stay reserved until the spend lands or the prune backstop releases them. /// /// Runs after the scan's spent-note reconcile (a landed spend's record /// was already dropped by the `mark_spent` hook, so anything still @@ -3244,8 +4121,6 @@ pub(super) async fn redrive_pending_identity_shields( pub(super) async fn redrive_pending_spends( sdk: &Arc, store: &Arc>, - persister: Option<&WalletPersister>, - wallet_id: WalletId, id: SubwalletId, recorded: &std::collections::HashSet<[u8; 32]>, cancel: &CancellationToken, @@ -3285,15 +4160,15 @@ pub(super) async fn redrive_pending_spends( Err(e) => { warn!( error = %e, - "redrive: stored transition failed to deserialize; dropping the record \ - (the prune backstop still frees the notes)" + "redrive: stored transition failed to deserialize; keeping the record \ + without re-broadcasting it (the prune backstop still frees the notes)" ); - if let Err(e) = store.write().await.clear_redrive(id, &redrive.activity_id) { - warn!(error = %e, "redrive: clear_redrive failed"); - } continue; } }; + let Some(attempts) = claim_redrive_attempt(store, id, &redrive).await else { + continue; + }; let broadcast_result = st.broadcast(sdk, None).await; let err_display = broadcast_result .as_ref() @@ -3302,7 +4177,6 @@ pub(super) async fn redrive_pending_spends( .unwrap_or_default(); match classify_redrive_broadcast(&broadcast_result) { RedriveBroadcastOutcome::Accepted => { - let attempts = bump_redrive_attempts_logged(store, id, &redrive.activity_id).await; info!( attempts, max = MAX_REDRIVE_ATTEMPTS, @@ -3310,12 +4184,11 @@ pub(super) async fn redrive_pending_spends( ); } RedriveBroadcastOutcome::AlreadyExecuted => { - // Success signal — but still consume an attempt: the scan + // Success signal, but the attempt still counts: the scan // normally confirms the landing and drops the record, and // if it lags, this arm must not re-broadcast unboundedly // on every pass. The cap parks the record for the scan / // prune passes to settle. - let attempts = bump_redrive_attempts_logged(store, id, &redrive.activity_id).await; info!( attempts, max = MAX_REDRIVE_ATTEMPTS, @@ -3323,37 +4196,10 @@ pub(super) async fn redrive_pending_spends( the notes spent" ); } - RedriveBroadcastOutcome::DefinitiveRejection => { - warn!( - error = %err_display, - "redrive: definitive consensus rejection; the spend can never execute — \ - releasing the reservation" - ); - { - let mut guard = store.write().await; - for n in &redrive.nullifiers { - // Also drops the redrive record via the - // clear_pending hook. - if let Err(e) = guard.clear_pending(id, n) { - warn!(error = %e, "redrive: clear_pending failed"); - } - } - } - record_activity_status_by_id( - store, - persister, - wallet_id, - id, - &redrive.activity_id, - ShieldedActivityStatus::Failed, - ) - .await; - } RedriveBroadcastOutcome::Inconclusive => { // `AlreadyExists` (still in a mempool after a lost-ACK // retry) or transport noise: inconclusive; counts toward // the cap. - let attempts = bump_redrive_attempts_logged(store, id, &redrive.activity_id).await; debug!( attempts, max = MAX_REDRIVE_ATTEMPTS, @@ -3376,7 +4222,7 @@ pub(super) async fn redrive_pending_spends( /// (Platform ran the transition and rejected it on its merits — see /// [`carries_consensus_rejection`]), means the spend never executed → /// [`PlatformWalletError::ShieldedBroadcastFailed`], and the caller -/// releases the note reservations via [`cancel_pending`]; +/// releases the note reservations via [`NoteReservation::cancel`]; /// - a no-verdict `broadcast()` failure (`AlreadyExists` proves the tx IS /// in the mempool after a lost-ACK retry; timeouts merely allow it) /// falls through to the result wait, which either proves execution or @@ -3413,18 +4259,52 @@ async fn broadcast_shielded_spend( operation: &'static str, wait: SpendResultWait, ) -> Result<(StateTransitionProofResult, ResponseMetadata), PlatformWalletError> { - match state_transition.broadcast(sdk, None).await { - Ok(()) => {} - Err(e) if broadcast_definitely_failed(&e) => { - return Err(crate::error::promote_address_nonce_error(&e) - .unwrap_or_else(|| PlatformWalletError::ShieldedBroadcastFailed(e.to_string()))); + broadcast_shielded_spend_classified(sdk, state_transition, operation, wait) + .await + .map_err(|failure| failure.error) +} + +/// A failed [`broadcast_shielded_spend_classified`]: the error it reports, and +/// whether the transition never left the wallet. +struct SpendFailure { + error: PlatformWalletError, + /// The broadcast proved nothing was delivered + /// ([`BroadcastDelivery::NeverDelivered`]): nothing holds the transition's + /// bytes but the wallet, so it can never execute. + never_delivered: bool, +} + +impl From for SpendFailure { + fn from(error: PlatformWalletError) -> Self { + Self { + error, + never_delivered: false, } - Err(e) => { + } +} + +/// [`broadcast_shielded_spend`], telling a failure whose transition never +/// left the wallet from one that may have reached the network. +async fn broadcast_shielded_spend_classified( + sdk: &Arc, + state_transition: &StateTransition, + operation: &'static str, + wait: SpendResultWait, +) -> Result<(StateTransitionProofResult, ResponseMetadata), SpendFailure> { + match broadcast_tracking_delivery(sdk, state_transition).await { + BroadcastDelivery::Admitted => {} + BroadcastDelivery::NeverDelivered(e) => { + return Err(SpendFailure { + error: definite_broadcast_failure(&e), + never_delivered: true, + }); + } + BroadcastDelivery::MaybeDelivered(e) => { warn!( operation, error = %e, - "Shielded spend broadcast returned no verdict; the transition may have been \ - admitted — falling through to the result wait" + "Shielded spend broadcast may have been delivered; falling through to the \ + result wait" ); } } @@ -3445,42 +4325,48 @@ async fn broadcast_shielded_spend( .await } }; - waited.map_err(|wait_err| classify_spend_wait_failure(operation, &wait_err)) + waited + .map_err(|wait_err| classify_spend_wait_failure(operation, &wait_err)) + .map_err(SpendFailure::from) +} + +/// The error a spend whose transition never left the wallet +/// ([`BroadcastDelivery::NeverDelivered`], refused before anything was sent) +/// is reported as: a failed broadcast. Its notes are released. Every +/// broadcast that reached an endpoint waits for its result instead, and +/// reads as unconfirmed when that wait fails. +fn definite_broadcast_failure(e: &dash_sdk::Error) -> PlatformWalletError { + PlatformWalletError::ShieldedBroadcastFailed(e.to_string()) } /// Classify a `wait_for_response` failure for an already-broadcast -/// shielded spend (see [`broadcast_shielded_spend`]). +/// shielded spend (see [`broadcast_shielded_spend`]): unconfirmed, whatever +/// it is, the notes staying reserved. /// -/// Only a consensus verdict ([`carries_consensus_rejection`]) proves -/// Platform executed the transition and rejected it on its merits — the -/// serialized consensus error is the verdict. DAPI encodes its own +/// A consensus verdict on the wait ([`carries_consensus_rejection`]) is no +/// proof either: the endpoint that sends it is unauthenticated, no proof or +/// quorum signature backs it, and one that kept or forwarded the signed +/// transition can answer with any verdict. DAPI also encodes its own /// wait-side failures (timeouts, internal errors; see /// `build_wait_for_state_transition_error_response` in rs-dapi) as -/// `StateTransitionBroadcastError`s with EMPTY consensus data, which -/// the SDK surfaces as `cause: None` — those are ambiguous, not -/// rejections, and must keep the note reservations. Erring this way is -/// the safe direction: misreading a rejection as ambiguous only delays -/// note release until the next sync or restart, while misreading a -/// timeout as a rejection re-frees notes whose nullifiers may already -/// be consumed on chain. +/// `StateTransitionBroadcastError`s with EMPTY consensus data. A spend +/// reported failed while it may still execute could be paid again with other +/// notes; one that never executes is released by the redrive and prune +/// passes once its anchor is pruned. fn classify_spend_wait_failure( operation: &'static str, wait_err: &dash_sdk::Error, ) -> PlatformWalletError { - if carries_consensus_rejection(wait_err) { - crate::error::promote_address_nonce_error(wait_err) - .unwrap_or_else(|| PlatformWalletError::ShieldedBroadcastFailed(wait_err.to_string())) - } else { - warn!( - operation, - error = %wait_err, - "Shielded spend broadcast accepted but result confirmation failed; \ - leaving the note reservations in place" - ); - PlatformWalletError::ShieldedSpendUnconfirmed { - operation, - reason: wait_err.to_string(), - } + warn!( + operation, + error = %wait_err, + rejected = carries_consensus_rejection(wait_err), + "Shielded spend broadcast but its result is unconfirmed; leaving the note reservations \ + in place" + ); + PlatformWalletError::ShieldedSpendUnconfirmed { + operation, + reason: wait_err.to_string(), } } @@ -3633,14 +4519,30 @@ mod shield_recipient_tests { #[cfg(test)] mod redrive_tests { use super::*; + use crate::wallet::identity::network::PaymentStarted; + use crate::wallet::shielded::activity::ShieldedActivityEntry; use crate::wallet::shielded::store::InMemoryShieldedStore; + use crate::wallet::shielded::FileBackedShieldedStore; + use dash_sdk::dpp::dashcore::Network; + use dash_sdk::dpp::prelude::CoreBlockHeight; + use dash_sdk::error::ContextProviderError; use dash_sdk::error::StateTransitionBroadcastError; + use dash_sdk::platform::ContextProvider; + use dash_sdk::sdk::AddressList; use dpp::consensus::state::shielded::nullifier_already_spent_error::NullifierAlreadySpentError; use dpp::consensus::state::state_error::StateError; use dpp::consensus::ConsensusError; - use dpp::platform_value::BinaryData; + use dpp::data_contract::{DataContract, TokenConfiguration}; use dpp::serialization::PlatformSerializable; + use dpp::shielded::SerializedAction; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::methods::PrivateDocumentCreateFromShieldedPoolTransitionMethodsV0; + use dpp::state_transition::private_document_create_from_shielded_pool_transition::PrivateDocumentCreateFromShieldedPoolTransition; use dpp::state_transition::shield_from_identity_transition::v0::ShieldFromIdentityTransitionV0; + use dpp::version::PlatformVersion; + use futures::{poll, FutureExt}; + use grovedb_commitment_tree::ProvingKey; + use std::str::FromStr; + use std::task::Poll; fn identity_shield_transition(identity_id: [u8; 32], nonce: u64) -> StateTransition { ShieldFromIdentityTransitionV0 { @@ -3674,6 +4576,7 @@ mod redrive_tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: true, + anchor_height: None, }; store.arm_redrive(id, archived.clone()).unwrap(); assert!(!has_unresolved_identity_shield(&store, wallet_id, identity_id).unwrap()); @@ -3724,6 +4627,7 @@ mod redrive_tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); @@ -3796,6 +4700,7 @@ mod redrive_tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .expect("redrive should arm"); @@ -3824,6 +4729,7 @@ mod redrive_tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .expect("redrive should arm"); @@ -3900,7 +4806,8 @@ mod redrive_tests { }); assert_eq!( classify_redrive_broadcast(&Err(rejection_err)), - RedriveBroadcastOutcome::DefinitiveRejection + RedriveBroadcastOutcome::Inconclusive, + "a refused re-broadcast does not prove the original never executes" ); let timeout = dash_sdk::Error::TimeoutReached( @@ -3934,6 +4841,7 @@ mod redrive_tests { attempts: MAX_REDRIVE_ATTEMPTS, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .expect("arm identity guard"); @@ -3957,44 +4865,1027 @@ mod redrive_tests { ); } + /// The production payment flow saves a private contact request payment + /// durably before its broadcast runs: intercepting the broadcast, a second + /// handle on the store's file reads its journal record and redrive record, + /// and the payment is marked started. A payment whose save fails is never + /// broadcast, is not marked started, and releases its notes. #[tokio::test] - async fn ordinary_redrive_pass_leaves_identity_guard_untouched() { - let sdk = Arc::new(dash_sdk::SdkBuilder::new_mock().build().expect("mock sdk")); - let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); - let wallet_id = [0x54; 32]; + async fn should_save_a_private_payment_durably_before_its_broadcast() { + use std::sync::atomic::{AtomicBool, Ordering}; + + let sdk = Arc::new(dash_sdk::Sdk::new_mock()); + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }; + let wallet_id = [0x68; 32]; let id = SubwalletId::new(wallet_id, 0); - let anchor = [0x76; 32]; - let activity_id = [0x98; 32]; - store - .write() - .await - .arm_redrive( - id, - PendingRedrive { - activity_id, - anchor, - nullifiers: vec![], - st_bytes: identity_shield_transition([0xBA; 32], 3) - .serialize_to_bytes() - .expect("serialize"), - attempts: 0, - identity_nonce_finalized: false, - identity_user_abandoned: false, + for save_fails in [false, true] { + let dir = tempfile::tempdir().expect("a directory"); + let path = dir.path().join("tree.sqlite"); + let store = Arc::new(RwLock::new( + FileBackedShieldedStore::open_path(&path, 100).expect("the store"), + )); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5F; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let (selected, _, _, reservation) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + if save_fails { + rusqlite::Connection::open(&path) + .expect("the store's file") + .execute_batch( + "CREATE TRIGGER reject_redrive_insert BEFORE INSERT ON \ + shielded_pending_spends BEGIN SELECT RAISE(FAIL, 'injected'); END;", + ) + .expect("the trigger"); + } + let transition = PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + Identifier::from([0x44; 32]), + "privateContactRequest".to_string(), + vec![BinaryData::new(vec![9; 589])], + vec![action.clone(), action.clone()], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition"); + let started = PaymentStarted::default(); + let broadcast = AtomicBool::new(false); + let submitted = submit_private_document_payment( + PrivateDocumentSubmission { + store: &store, + persister: None, + wallet_id, + id, + pending_entry: Some(ShieldedActivityEntry { + id: [0x77; 32], + kind: ShieldedActivityKind::ShieldedSpend, + direction: ShieldedDirection::Out, + amount: 0, + fee: Some(0), + counterparty: None, + memo: None, + block_height: None, + status: ShieldedActivityStatus::Pending, + created_at_ms: 1, + min_note_position: None, + note_cmxs: vec![], + spent_nullifiers: vec![], + }), + selected_notes: &selected, + reservation, + state_transition: transition, + anchor: [7; 32], + anchor_height: 0, + private_contact: Some(payment), + journaled: Some(&[[9; 32]]), + payment_started: &started, + }, + |_| { + broadcast.store(true, Ordering::SeqCst); + let durable = + FileBackedShieldedStore::open_path(&path, 100).expect("a second handle"); + assert_eq!( + durable + .private_payment_journal(wallet_id) + .expect("the journal") + .len(), + 1, + "the journal record is on the file" + ); + assert_eq!( + durable + .private_contact_redrives_for_wallet(wallet_id) + .expect("the records") + .len(), + 1, + "the redrive record and its association are on the file" + ); + assert!(started.is_marked(), "marked once saved"); + std::future::ready(Err(SpendFailure { + error: PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: "intercepted".to_string(), + }, + never_delivered: false, + })) }, ) - .expect("arm identity guard"); - let recorded = [anchor].into_iter().collect(); + .await; + let unspent = store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .len(); + if save_fails { + assert!( + matches!(submitted, Err(PlatformWalletError::ShieldedStoreError(_))), + "{submitted:?}" + ); + assert!(!broadcast.load(Ordering::SeqCst), "never broadcast"); + assert!(!started.is_marked(), "never marked"); + assert_eq!(unspent, 1, "the notes are released"); + assert!(store + .read() + .await + .private_payment_journal(wallet_id) + .expect("the journal") + .is_empty()); + } else { + assert!( + matches!( + submitted, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{submitted:?}" + ); + assert!(broadcast.load(Ordering::SeqCst)); + assert_eq!(unspent, 0, "the notes stay reserved"); + } + } + } + + /// A payment dropped while its recovery save waits for the store has saved + /// and broadcast nothing, and its note guard releases the notes once the + /// store frees: the notes are handed over only after the save. + #[tokio::test] + async fn should_release_the_notes_of_a_payment_dropped_while_its_save_waits() { + use std::future::Future; + use std::task::{Context, Poll, Waker}; + + let sdk = Arc::new(dash_sdk::Sdk::new_mock()); + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + let wallet_id = [0x69; 32]; + let id = SubwalletId::new(wallet_id, 0); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5E; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let (selected, _, _, reservation) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + let transition = PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + Identifier::from([0x44; 32]), + "privateContactRequest".to_string(), + vec![BinaryData::new(vec![9; 589])], + vec![action.clone(), action], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition"); + let started = PaymentStarted::default(); + let mut cx = Context::from_waker(Waker::noop()); + + let held = store.write().await; + let mut submitted = Box::pin(submit_private_document_payment( + PrivateDocumentSubmission { + store: &store, + persister: None, + wallet_id, + id, + pending_entry: Some(ShieldedActivityEntry { + id: [0x78; 32], + kind: ShieldedActivityKind::ShieldedSpend, + direction: ShieldedDirection::Out, + amount: 0, + fee: Some(0), + counterparty: None, + memo: None, + block_height: None, + status: ShieldedActivityStatus::Pending, + created_at_ms: 1, + min_note_position: None, + note_cmxs: vec![], + spent_nullifiers: vec![], + }), + selected_notes: &selected, + reservation, + state_transition: transition, + anchor: [7; 32], + anchor_height: 0, + private_contact: Some(PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }), + journaled: Some(&[[9; 32]]), + payment_started: &started, + }, + |_| async { panic!("never broadcast") }, + )); + // The submission waits for the store before its anchor step. + assert!(submitted.as_mut().poll(&mut cx).is_pending()); + // Freed, the lock goes to the submission first; this request queues + // behind it, so the submission's next one, the save, waits for it. + drop(held); + let mut ours = Box::pin(store.write()); + assert!(ours.as_mut().poll(&mut cx).is_pending()); + assert!(submitted.as_mut().poll(&mut cx).is_pending()); + let Poll::Ready(held) = ours.as_mut().poll(&mut cx) else { + panic!("the store is ours once the submission's anchor step is done"); + }; + // Dropped while the save waits. + drop(submitted); + assert!(!started.is_marked(), "nothing was saved"); + drop(held); + for _ in 0..50 { + if !store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .is_empty() + { + break; + } + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + assert_eq!( + store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .len(), + 1, + "the guard released the notes" + ); + assert!(store + .read() + .await + .private_payment_journal(wallet_id) + .expect("the journal") + .is_empty()); + } + + /// A stale spend attempt's detached cleanup, delayed until after an + /// account purge and rebind (or the wallet's replacement) let a newer + /// attempt reserve the same notes and save its payment, releases nothing: + /// the newer attempt's notes stay reserved, and its redrive record, + /// association and journal record stay. + #[tokio::test] + async fn should_not_release_a_newer_attempt_s_notes_from_a_stale_cleanup() { + let sdk = Arc::new(dash_sdk::Sdk::new_mock()); + let wallet_id = [0x6A; 32]; + let id = SubwalletId::new(wallet_id, 0); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5D; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }; + for wallet_replaced in [false, true] { + let dir = tempfile::tempdir().expect("a directory"); + let store = Arc::new(RwLock::new( + FileBackedShieldedStore::open_path(dir.path().join("tree.sqlite"), 100) + .expect("the store"), + )); + store.write().await.save_note(id, ¬e).expect("a note"); + let (_, _, _, stale) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + // Dropped while the store is busy: its cleanup waits on a task. + let mut held = store.write().await; + drop(stale); + if wallet_replaced { + held.purge_wallet(wallet_id).expect("the wallet's purge"); + } else { + held.purge_subwallet(id).expect("the account's purge"); + } + held.save_note(id, ¬e).expect("the note again"); + assert!(held + .mark_pending_by(id, ¬e.nullifier, u64::MAX) + .expect("reserved by the newer attempt")); + held.arm_private_contact_payment( + id, + PendingRedrive { + activity_id: [0x79; 32], + anchor: [7; 32], + nullifiers: vec![note.nullifier], + st_bytes: vec![0xAB; 96], + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + payment, + &[[0xC4; 32]], + 0, + ) + .expect("the newer payment saved"); + drop(held); + // The stale cleanup gets the store now. + for _ in 0..10 { + tokio::task::yield_now().await; + tokio::time::sleep(std::time::Duration::from_millis(5)).await; + } + let store = store.read().await; + let context = format!("wallet replaced: {wallet_replaced}"); + assert!( + store.get_unspent_notes(id).expect("notes").is_empty(), + "{context}: the newer attempt's note stays reserved" + ); + assert_eq!( + store.pending_redrives(id).expect("records").len(), + 1, + "{context}" + ); + assert_eq!( + store + .private_contact_redrives_for_wallet(wallet_id) + .expect("associations") + .len(), + 1, + "{context}" + ); + assert_eq!( + store + .private_payment_journal(wallet_id) + .expect("the journal") + .len(), + 1, + "{context}" + ); + } + } + + /// A private contact request payment's journal record outlives its notes' + /// and redrive record whatever the spend's outcome, except a transition + /// that provably never left the wallet. Settled unconfirmed (its + /// broadcast may have been delivered and the result wait failed), the + /// notes stay reserved and the record, association and journal record + /// survive a reopen. Rejected after delivery, the notes and record are + /// released, the journal record stays. Never delivered, the record and + /// the journal record are retired together and the payment reads as not + /// reaching the network; unless the redrive pass claimed a re-broadcast + /// of the record meanwhile, when it may leave the wallet after all and is + /// settled as unconfirmed, keeping everything. + #[tokio::test] + async fn should_keep_a_private_payment_s_journal_record_unless_it_never_left() { + let sdk = Arc::new(dash_sdk::Sdk::new_mock()); + let action = SerializedAction { + nullifier: [1; 32], + rk: [2; 32], + cmx: [3; 32], + encrypted_note: vec![4; 216], + cv_net: [5; 32], + spend_auth_sig: [6; 64], + }; + let entry = BinaryData::new(vec![9; 589]); + let commitment = [9u8; 32]; + let transition = PrivateDocumentCreateFromShieldedPoolTransition::try_from_bundle( + Identifier::from([0x44; 32]), + "privateContactRequest".to_string(), + vec![entry], + vec![action.clone(), action], + 0, + [7; 32], + vec![8; 64], + [9; 64], + PlatformVersion::latest(), + ) + .expect("a type 29 transition"); + let payment = PrivateContactPayment { + sender: [0xB1; 32], + recipient: [0xA1; 32], + account_reference: 5, + }; + let wallet_id = [0x67; 32]; + let id = SubwalletId::new(wallet_id, 0); + let failure = |outcome: &str| match outcome { + "unconfirmed" => SpendFailure { + error: PlatformWalletError::ShieldedSpendUnconfirmed { + operation: "private document create from shielded pool", + reason: "the result wait timed out".to_string(), + }, + never_delivered: false, + }, + "rejected after delivery" => SpendFailure { + error: PlatformWalletError::ShieldedBroadcastFailed( + "rejected on its merits".to_string(), + ), + never_delivered: false, + }, + _ => SpendFailure { + error: PlatformWalletError::ShieldedBroadcastFailed( + "refused at the door".to_string(), + ), + never_delivered: true, + }, + }; + for outcome in [ + "unconfirmed", + "rejected after delivery", + "never delivered", + "never delivered, re-broadcast claimed", + ] { + let dir = tempfile::tempdir().expect("a directory"); + let path = dir.path().join("tree.sqlite"); + let store = Arc::new(RwLock::new( + FileBackedShieldedStore::open_path(&path, 100).expect("the store"), + )); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5F; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let (selected, _, _, reservation) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + let started = PaymentStarted::default(); + arm_redrive_before_broadcast( + &store, + id, + Some([0x77; 32]), + [7; 32], + selected.iter().map(|note| note.nullifier).collect(), + &transition, + "private document create from shielded pool", + Some(payment), + Some(&started), + 0, + ) + .await + .expect("saved before the broadcast"); + assert_eq!( + store + .read() + .await + .private_payment_journal(wallet_id) + .expect("journal") + .len(), + 1, + "{outcome}: journaled before the broadcast" + ); + + if outcome == "never delivered, re-broadcast claimed" { + store + .write() + .await + .bump_redrive_attempts(id, &[0x77; 32]) + .expect("claimed"); + } + let settled = settle_private_document_payment( + &store, + None, + wallet_id, + id, + &None, + Some([0x77; 32]), + reservation, + &selected, + Some(&[commitment]), + &started, + Err(failure(outcome)), + ) + .await; + assert!(settled.is_err(), "{outcome}: {settled:?}"); + let unspent = store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .len(); + drop(store); + let reopened = FileBackedShieldedStore::open_path(&path, 100).expect("reopened"); + let journal = reopened + .private_payment_journal(wallet_id) + .expect("journal"); + let saved = reopened + .private_contact_redrives_for_wallet(wallet_id) + .expect("listed"); + match outcome { + "unconfirmed" | "never delivered, re-broadcast claimed" => { + assert!( + matches!( + settled, + Err(PlatformWalletError::ShieldedSpendUnconfirmed { .. }) + ), + "{outcome}: {settled:?}" + ); + assert_eq!(unspent, 0, "the notes stay reserved"); + assert_eq!(saved.len(), 1, "the record and its association stay"); + assert_eq!(journal.len(), 1, "the journal record stays"); + assert!(started.may_have_reached_network()); + } + "rejected after delivery" => { + assert_eq!(unspent, 1, "the notes are released"); + assert!(saved.is_empty(), "the record goes with them"); + assert_eq!(journal.len(), 1, "the journal record stays"); + assert_eq!(journal[0].commitment, commitment); + assert_eq!(journal[0].payment, payment); + assert!(started.may_have_reached_network()); + } + _ => { + assert_eq!(unspent, 1, "the notes are released"); + assert!(saved.is_empty(), "the record goes with them"); + assert!(journal.is_empty(), "the journal record is discharged"); + assert!(!started.may_have_reached_network()); + } + } + } + } + + /// A spend that failed before anything was broadcast, here a private document payment + /// whose anchor fetch the mock refuses, and is dropped while its rollback waits for the + /// store lock another task holds, still releases its notes: the guard owns them until the + /// rollback has cleared them. + #[tokio::test] + async fn should_release_the_notes_of_a_spend_dropped_during_its_failure_rollback() { + struct UnreachableProver; + impl OrchardProver for UnreachableProver { + fn proving_key(&self) -> &ProvingKey { + panic!("nothing is proved") + } + } - redrive_pending_spends( + let sdk = Arc::new( + dash_sdk::SdkBuilder::new_mock() + .with_version(PlatformVersion::latest()) + .build() + .expect("a mock SDK"), + ); + let wallet_id = [0x65; 32]; + let id = SubwalletId::new(wallet_id, 0); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5E; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let keys = OrchardKeySet::from_seed(&[0x42; 64], Network::Testnet, 0).expect("keys"); + let started = PaymentStarted::default(); + + // The spend waits for the store behind this task; this task's next acquisition waits + // behind the spend, so the lock passes to it right after the spend's reservation. + let held = store.write().await; + let mut spend = Box::pin(private_document_create_from_pool_with_metadata( &sdk, &store, None, wallet_id, + &keys, + 0, + Identifier::from([0x44; 32]), + "privateContactRequest", + vec![BinaryData::new(vec![1; 589])], + None, + &started, + &UnreachableProver, + )); + assert!(poll!(&mut spend).is_pending(), "waits to reserve"); + let mut again = Box::pin(store.write()); + assert!(poll!(&mut again).is_pending(), "queued behind the spend"); + drop(held); + // Reserved, refused by the anchor fetch, and waiting to roll back behind `again`. + assert!( + poll!(&mut spend).is_pending(), + "the rollback waits for the store" + ); + let Poll::Ready(contended) = poll!(&mut again) else { + panic!("the lock passes on after the reservation"); + }; + assert!( + contended.get_unspent_notes(id).expect("notes").is_empty(), + "reserved while the rollback waits" + ); + drop(spend); + drop(contended); + for _ in 0..100 { + if !store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .is_empty() + { + break; + } + tokio::task::yield_now().await; + } + assert_eq!( + store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .iter() + .map(|note| note.nullifier) + .collect::>(), + vec![note.nullifier], + "the note is spendable again" + ); + assert!(!started.is_marked()); + } + + /// A reservation dropped while the store is locked is released once the + /// lock frees, on a spawned task; one handed over stays. + #[tokio::test] + async fn should_release_a_dropped_reservation_once_the_store_frees() { + let sdk = Arc::new(dash_sdk::Sdk::new_mock()); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let id = SubwalletId::new([0x64; 32], 0); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5B; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let unspent = || async { + store + .read() + .await + .get_unspent_notes(id) + .expect("notes") + .len() + }; + + let (_, _, _, reservation) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + assert_eq!(unspent().await, 0); + let held = store.write().await; + drop(reservation); + drop(held); + for _ in 0..100 { + if unspent().await == 1 { + break; + } + tokio::task::yield_now().await; + } + assert_eq!(unspent().await, 1, "released once the lock freed"); + + let (_, _, _, mut kept) = + reserve_notes_covering(&sdk, &store, id, 1, 2, ShieldedFeeKind::Base) + .await + .expect("reserved"); + kept.hand_over(); + drop(kept); + assert_eq!(unspent().await, 0, "the spend owns it"); + } + + /// A spend dropped while it fetches the anchors, before its notes have an + /// owner that frees them, leaves them spendable again: here a private + /// document payment whose anchor fetch goes to a node that accepts the + /// connection and never answers. + #[tokio::test] + async fn should_release_the_notes_of_a_spend_dropped_during_its_anchor_fetch() { + struct NoKeys; + impl ContextProvider for NoKeys { + fn get_data_contract( + &self, + _id: &Identifier, + _platform_version: &PlatformVersion, + ) -> Result>, ContextProviderError> { + Ok(None) + } + fn get_token_configuration( + &self, + _token_id: &Identifier, + ) -> Result, ContextProviderError> { + Ok(None) + } + fn get_quorum_public_key( + &self, + _quorum_type: u32, + _quorum_hash: [u8; 32], + _core_chain_locked_height: u32, + ) -> Result<[u8; 48], ContextProviderError> { + Err(ContextProviderError::Generic("no quorum keys".to_string())) + } + fn get_platform_activation_height( + &self, + ) -> Result { + Ok(1) + } + } + struct UnreachableProver; + impl OrchardProver for UnreachableProver { + fn proving_key(&self) -> &ProvingKey { + panic!("nothing is proved") + } + } + + /// A loopback node that accepts connections and never answers. Dropped, + /// even while a failed assertion unwinds, it stops accepting, closes + /// what it accepted and joins its thread. + struct SilentNode { + address: std::net::SocketAddr, + shutdown: Arc, + thread: Option>, + } + impl SilentNode { + fn start() -> Self { + let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("a loopback port"); + listener.set_nonblocking(true).expect("non-blocking accept"); + let address = listener.local_addr().expect("its address"); + let shutdown = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let stop = Arc::clone(&shutdown); + let thread = std::thread::spawn(move || { + let mut held = Vec::new(); + while !stop.load(std::sync::atomic::Ordering::Acquire) { + match listener.accept() { + Ok((stream, _)) => held.push(stream), + Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => { + std::thread::sleep(std::time::Duration::from_millis(5)); + } + Err(_) => break, + } + } + drop(held); + }); + Self { + address, + shutdown, + thread: Some(thread), + } + } + } + impl Drop for SilentNode { + fn drop(&mut self) { + self.shutdown + .store(true, std::sync::atomic::Ordering::Release); + if let Some(thread) = self.thread.take() { + let _ = thread.join(); + } + } + } + + let node = SilentNode::start(); + let address = node.address; + let sdk = Arc::new( + dash_sdk::SdkBuilder::new( + AddressList::from_str(&format!("http://{address}")).expect("an address"), + ) + .with_network(Network::Testnet) + .with_version(PlatformVersion::latest()) + .with_context_provider(NoKeys) + .build() + .expect("a network SDK"), + ); + + let wallet_id = [0x63; 32]; + let id = SubwalletId::new(wallet_id, 0); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let note = ShieldedNote { + position: 0, + cmx: [1; 32], + nullifier: [0x5A; 32], + block_height: 1, + is_spent: false, + value: 1_000_000_000_000, + note_data: vec![], + }; + store.write().await.save_note(id, ¬e).expect("a note"); + let keys = OrchardKeySet::from_seed(&[0x42; 64], Network::Testnet, 0).expect("keys"); + let started = PaymentStarted::default(); + + let paid = tokio::time::timeout( + std::time::Duration::from_millis(500), + private_document_create_from_pool_with_metadata( + &sdk, + &store, + None, + wallet_id, + &keys, + 0, + Identifier::from([0x44; 32]), + "privateContactRequest", + vec![BinaryData::new(vec![1; 589])], + None, + &started, + &UnreachableProver, + ), + ) + .await; + assert!(paid.is_err(), "dropped during the anchor fetch: {paid:?}"); + assert!(!started.is_marked()); + let unspent = store.read().await.get_unspent_notes(id).expect("notes"); + assert_eq!( + unspent + .iter() + .map(|note| note.nullifier) + .collect::>(), + vec![note.nullifier], + "the note is spendable again" + ); + } + + /// A private payment is marked started only once its record is saved: + /// dropped while the save waits for the store, the spend has saved + /// nothing and is not marked, so its contact request's reservation is + /// released rather than kept for a payment that never left. Once the + /// lock frees, a save marks it. + #[tokio::test] + async fn should_mark_a_payment_started_only_once_its_record_is_saved() { + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let id = SubwalletId::new([0x62; 32], 0); + let transition = identity_shield_transition([0xBB; 32], 4); + let started = PaymentStarted::default(); + + let held = store.write().await; + let save = arm_redrive_before_broadcast( + &store, id, - &recorded, - &CancellationToken::new(), + Some([0x78; 32]), + [7; 32], + vec![[2; 32]], + &transition, + "test spend", + None, + Some(&started), + 0, + ); + let mut save = Box::pin(save); + assert!( + (&mut save).now_or_never().is_none(), + "the save waits for the store" + ); + drop(save); + drop(held); + assert!(!started.is_marked(), "dropped before its record was saved"); + assert!(store + .read() + .await + .pending_redrives(id) + .expect("redrives") + .is_empty()); + + arm_redrive_before_broadcast( + &store, + id, + Some([0x78; 32]), + [7; 32], + vec![[2; 32]], + &transition, + "test spend", + None, + Some(&started), + 0, + ) + .await + .expect("saved"); + assert!(started.is_marked()); + assert_eq!( + store + .read() + .await + .pending_redrives(id) + .expect("redrives") + .len(), + 1 + ); + } + + /// A spend's record is saved before its broadcast, keyed by its + /// activity entry; one with no entry to key it by is refused, nothing + /// saved. + #[tokio::test] + async fn should_save_a_spends_record_before_its_broadcast() { + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let id = SubwalletId::new([0x61; 32], 0); + let transition = identity_shield_transition([0xBA; 32], 3); + + let refused = arm_redrive_before_broadcast( + &store, + id, + None, + [7; 32], + vec![[1; 32]], + &transition, + "test spend", + None, + None, + 0, ) .await; + assert!( + matches!(refused, Err(PlatformWalletError::ShieldedBuildError(_))), + "{refused:?}" + ); + assert!(store + .read() + .await + .pending_redrives(id) + .expect("redrives") + .is_empty()); + + arm_redrive_before_broadcast( + &store, + id, + Some([0x77; 32]), + [7; 32], + vec![[1; 32]], + &transition, + "test spend", + None, + None, + 0, + ) + .await + .expect("saved"); + let saved = store.read().await.pending_redrives(id).expect("redrives"); + assert_eq!(saved.len(), 1); + assert_eq!(saved[0].activity_id, [0x77; 32]); + assert_eq!(saved[0].nullifiers, vec![[1; 32]]); + } + + #[tokio::test] + async fn ordinary_redrive_pass_leaves_identity_guard_untouched() { + let sdk = Arc::new(dash_sdk::SdkBuilder::new_mock().build().expect("mock sdk")); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let wallet_id = [0x54; 32]; + let id = SubwalletId::new(wallet_id, 0); + let anchor = [0x76; 32]; + let activity_id = [0x98; 32]; + store + .write() + .await + .arm_redrive( + id, + PendingRedrive { + activity_id, + anchor, + nullifiers: vec![], + st_bytes: identity_shield_transition([0xBA; 32], 3) + .serialize_to_bytes() + .expect("serialize"), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }, + ) + .expect("arm identity guard"); + let recorded = [anchor].into_iter().collect(); + + redrive_pending_spends(&sdk, &store, id, &recorded, &CancellationToken::new()).await; let retained = store.read().await.pending_redrives(id).expect("guard"); assert_eq!(retained.len(), 1); @@ -4008,9 +5899,10 @@ mod redrive_tests { /// no broadcast expectation, so any attempt would error into the /// inconclusive arm and bump the counter): a pruned anchor belongs /// to the release backstop, an exhausted budget stays parked, and a - /// corrupt stored transition is dropped so it can't wedge the pass. + /// corrupt stored transition is kept without a broadcast, as it may be + /// the only evidence of a payment that can still execute. #[tokio::test] - async fn redrive_skips_pruned_and_exhausted_and_drops_garbage() { + async fn should_skip_pruned_exhausted_and_undecodable_redrives_keeping_each() { let sdk = Arc::new(dash_sdk::SdkBuilder::new_mock().build().expect("mock sdk")); let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); let wallet_id = [5u8; 32]; @@ -4024,6 +5916,7 @@ mod redrive_tests { attempts, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; { let mut guard = store.write().await; @@ -4033,31 +5926,22 @@ mod redrive_tests { guard .arm_redrive(id, rec(2, 11, MAX_REDRIVE_ATTEMPTS)) .unwrap(); - // Recorded anchor, budget left, garbage bytes → dropped - // before any broadcast. + // Recorded anchor, budget left, garbage bytes → kept, with no + // broadcast. guard.arm_redrive(id, rec(3, 12, 0)).unwrap(); } let recorded: std::collections::HashSet<[u8; 32]> = [[11u8; 32], [12u8; 32]].into_iter().collect(); - redrive_pending_spends( - &sdk, - &store, - None, - wallet_id, - id, - &recorded, - &CancellationToken::new(), - ) - .await; + redrive_pending_spends(&sdk, &store, id, &recorded, &CancellationToken::new()).await; let left = store.read().await.pending_redrives(id).unwrap(); let ids: Vec<[u8; 32]> = left.iter().map(|r| r.activity_id).collect(); assert!(ids.contains(&[1u8; 32]), "pruned-anchor record left alone"); assert!(ids.contains(&[2u8; 32]), "exhausted record left alone"); assert!( - !ids.contains(&[3u8; 32]), - "corrupt record dropped without a broadcast attempt" + ids.contains(&[3u8; 32]), + "corrupt record kept without a broadcast attempt" ); assert_eq!( left.iter().map(|r| r.attempts).max(), @@ -4065,6 +5949,93 @@ mod redrive_tests { "no attempt counters were bumped — nothing touched the network" ); } + + /// A record retired while the redrive pass holds a snapshot that still + /// has it is not re-broadcast: the pass claims each re-broadcast under + /// the store write lock, from the record as it is then. Here the pass is + /// held at its claim of an earlier record while the retirement queues for + /// the lock, so it reaches the retired record with its stale snapshot. + #[tokio::test] + async fn should_not_rebroadcast_a_record_retired_after_the_pass_read_it() { + use dash_sdk::platform::transition::broadcast_request::BroadcastRequestForStateTransition; + use dpp::serialization::PlatformSerializable; + use std::future::Future; + use std::task::{Context, Waker}; + + let mut sdk = dash_sdk::SdkBuilder::new_mock().build().expect("mock sdk"); + let store = Arc::new(RwLock::new(InMemoryShieldedStore::new())); + let id = SubwalletId::new([0x5D; 32], 0); + let anchor = [0x6D; 32]; + let earlier = identity_shield_transition([0xA1; 32], 3); + let retired = identity_shield_transition([0xA2; 32], 4); + let record = |activity: u8, transition: &StateTransition| PendingRedrive { + activity_id: [activity; 32], + anchor, + nullifiers: vec![[activity ^ 0xFF; 32]], + st_bytes: transition.serialize_to_bytes().expect("serializes"), + attempts: 0, + identity_nonce_finalized: false, + identity_user_abandoned: false, + anchor_height: None, + }; + { + let mut guard = store.write().await; + guard.arm_redrive(id, record(1, &earlier)).expect("armed"); + guard.arm_redrive(id, record(2, &retired)).expect("armed"); + } + let recorded: std::collections::HashSet<[u8; 32]> = [anchor].into_iter().collect(); + let sdk_handle = Arc::new(sdk.clone()); + let cancel = CancellationToken::new(); + + let mut cx = Context::from_waker(Waker::noop()); + let held = store.read().await; + let mut pass = Box::pin(redrive_pending_spends( + &sdk_handle, + &store, + id, + &recorded, + &cancel, + )); + assert!( + pass.as_mut().poll(&mut cx).is_pending(), + "the pass read both records and waits to claim the first" + ); + let mut retire = Box::pin(async { + store + .write() + .await + .retire_undelivered_spend(id, &[2; 32], &[]) + }); + assert!( + retire.as_mut().poll(&mut cx).is_pending(), + "the retirement queues behind the claim" + ); + drop(held); + let ((), retirement) = tokio::join!(pass, retire); + assert!( + retirement.expect("retired"), + "no re-broadcast was claimed for it" + ); + + let request = |transition: &StateTransition| { + transition + .broadcast_request_for_state_transition() + .expect("serializes") + }; + assert!( + sdk.mock().request_count(&request(&earlier)).await > 0, + "the earlier record is re-broadcast" + ); + assert_eq!( + sdk.mock().request_count(&request(&retired)).await, + 0, + "the retired record is not" + ); + let left = store.read().await.pending_redrives(id).expect("records"); + assert_eq!(left.len(), 1); + assert_eq!(left[0].activity_id, [1; 32]); + assert_eq!(left[0].attempts, 1, "claimed once, before its broadcast"); + } } #[cfg(test)] @@ -4083,16 +6054,18 @@ mod classify_spend_wait_failure_tests { }) } + /// A broadcast error carrying a consensus cause comes from an + /// unauthenticated endpoint, with no proof behind it: the spend stays + /// unconfirmed and its reservation stays. #[test] - fn consensus_cause_is_a_definitive_rejection() { + fn should_read_a_wait_stage_consensus_verdict_as_unconfirmed() { let cause = ConsensusError::BasicError(BasicError::ProtocolVersionParsingError( ProtocolVersionParsingError::new("bad version".to_string()), )); let err = classify_spend_wait_failure("unshield", &broadcast_err(Some(cause))); assert!( - matches!(err, PlatformWalletError::ShieldedBroadcastFailed(_)), - "a broadcast error carrying a consensus cause means Platform ran and \ - rejected the transition; the caller may release the reservation" + matches!(err, PlatformWalletError::ShieldedSpendUnconfirmed { .. }), + "{err:?}" ); } @@ -4144,18 +6117,19 @@ mod classify_spend_wait_failure_tests { dash_sdk::Error::Protocol(dpp::ProtocolError::ConsensusError(Box::new(cause))) } - /// A CheckTx consensus rejection surfacing from `broadcast()` IS - /// definitive: the transition was evaluated and refused, it sits in no - /// mempool, and the caller may release the reservations and retry. + /// A consensus rejection surfacing from `broadcast()` stays definitive + /// for the shields into the pool that classify with + /// `broadcast_definitely_failed`, which move funds to the wallet's own + /// notes. On a note spend's wait the same verdict reads as unconfirmed: + /// no proof backs it. #[test] - fn broadcast_consensus_rejection_is_definitive() { + fn should_read_a_consensus_rejection_as_definitive_only_for_shields_into_the_pool() { assert!(broadcast_definitely_failed(&consensus_metadata_rejection())); - // The same verdict shape is definitive on the wait stage too. let err = classify_spend_wait_failure("transfer", &consensus_metadata_rejection()); - assert!(matches!( - err, - PlatformWalletError::ShieldedBroadcastFailed(_) - )); + assert!( + matches!(err, PlatformWalletError::ShieldedSpendUnconfirmed { .. }), + "{err:?}" + ); } fn grpc_err(code: dash_sdk::dapi_grpc::tonic::Code) -> dash_sdk::Error { @@ -4248,11 +6222,11 @@ mod classify_spend_wait_failure_tests { assert!(!carries_consensus_rejection(&wrapped)); } - /// A nonce rejection wrapped in the retry envelope must reach - /// `promote_address_nonce_error` and surface as the typed - /// `AddressNonceMismatch`, not fall through to `ShieldedSpendUnconfirmed`. + /// A nonce rejection on a note spend's wait, wrapped in the retry + /// envelope, is an unauthenticated verdict too: unconfirmed, the notes + /// kept, never a typed refusal a host could retry on. #[test] - fn wrapped_nonce_rejection_promotes_to_typed_mismatch() { + fn should_read_a_wrapped_nonce_rejection_on_the_wait_as_unconfirmed() { use dpp::address_funds::PlatformAddress; use dpp::consensus::state::address_funds::AddressInvalidNonceError; use dpp::consensus::state::state_error::StateError; @@ -4264,18 +6238,11 @@ mod classify_spend_wait_failure_tests { let inner = dash_sdk::Error::Protocol(dpp::ProtocolError::ConsensusError(Box::new(cause))); let wrapped = dash_sdk::Error::NoAvailableAddressesToRetry(Box::new(inner)); - match classify_spend_wait_failure("withdraw", &wrapped) { - PlatformWalletError::AddressNonceMismatch { - address: got, - provided_nonce, - expected_nonce, - } => { - assert_eq!(got, address); - assert_eq!(provided_nonce, 7); - assert_eq!(expected_nonce, 8); - } - other => panic!("expected AddressNonceMismatch, got {other:?}"), - } + let err = classify_spend_wait_failure("withdraw", &wrapped); + assert!( + matches!(err, PlatformWalletError::ShieldedSpendUnconfirmed { .. }), + "{err:?}" + ); } } diff --git a/packages/rs-platform-wallet/src/wallet/shielded/store.rs b/packages/rs-platform-wallet/src/wallet/shielded/store.rs index 8efd0acfdcb..fc2f906faf9 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/store.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/store.rs @@ -127,11 +127,12 @@ pub struct ShieldedOutgoingNote { } /// A pending reservation that carries a recorded anchor: -/// `(nullifier, anchor, activity_id)`. Yielded by -/// [`ShieldedStore::stale_pending_spends`] for the sync reconcile, -/// which releases the reservation once `anchor` is no longer in -/// Platform's recorded set (the spend can then never execute). -pub type StalePendingSpend = ([u8; 32], [u8; 32], Option<[u8; 32]>); +/// `(nullifier, anchor, activity_id, anchor_height)`, `anchor_height` being a +/// proved height at which `anchor` was in Platform's recorded set, when known. +/// Yielded by [`ShieldedStore::stale_pending_spends`] for the sync +/// reconcile, which releases the reservation once `anchor` is proven pruned +/// (the spend can then never execute). +pub type StalePendingSpend = ([u8; 32], [u8; 32], Option<[u8; 32]>, Option); /// An unresolved signed transition and its durable retry guard: the signed /// transition bytes plus everything the sync-time re-drive needs to @@ -162,6 +163,14 @@ pub struct PendingRedrive { pub st_bytes: Vec, /// Re-broadcast attempts made so far. pub attempts: u32, + /// A proved height at which [`Self::anchor`] was in Platform's recorded + /// anchor set: the height of the set the spend's anchor was read from, or + /// of a later set seen holding it + /// ([`ShieldedStore::note_anchor_recorded`]). A set proved at or above it + /// that lacks the anchor shows it pruned rather than newer. `None` for a + /// record saved without it, whose anchor is not proven pruned until a set + /// is seen holding it. + pub anchor_height: Option, /// A proven identity nonce has made this exact debit impossible to execute /// again. Stop broadcasting, but keep its unresolved-payment guard until /// the scan confirms the outputs. Always false for note-spend redrives. @@ -173,6 +182,153 @@ pub struct PendingRedrive { pub identity_user_abandoned: bool, } +/// The DashPay private contact request a saved spend pays for: which identity +/// sends it to which, and its account reference. Kept with the spend's +/// record, so a restart restores the pair's reservation from the record +/// without the keys that open its entry. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct PrivateContactPayment { + /// The identity sending the request. + pub sender: [u8; 32], + /// The identity it is to. + pub recipient: [u8; 32], + /// The request's account reference, whose version tells a rotation from + /// the request it rotates. + pub account_reference: u32, +} + +impl PrivateContactPayment { + /// The bytes the file store keeps it as: sender, recipient, then the + /// account reference little-endian. + pub(super) fn to_bytes(self) -> [u8; 68] { + let mut bytes = [0u8; 68]; + bytes[..32].copy_from_slice(&self.sender); + bytes[32..64].copy_from_slice(&self.recipient); + bytes[64..].copy_from_slice(&self.account_reference.to_le_bytes()); + bytes + } + + /// The payment [`Self::to_bytes`] wrote, or `None` for other bytes. + pub(super) fn from_bytes(bytes: &[u8]) -> Option { + if bytes.len() != 68 { + return None; + } + Some(Self { + sender: bytes.get(..32)?.try_into().ok()?, + recipient: bytes.get(32..64)?.try_into().ok()?, + account_reference: u32::from_le_bytes(bytes.get(64..)?.try_into().ok()?), + }) + } +} + +/// One private contact request payment the wallet may have broadcast, as the +/// shielded store's private payment journal keeps it: the request it pays +/// for, the commitment of the entry it appends and the nullifiers its spend +/// consumes. Saved durably before the payment can reach the network +/// ([`ShieldedStore::arm_private_contact_payment`]) and kept until the +/// payment provably resolved ([`ShieldedStore::discharge_private_payment`]), +/// whatever happens to the spend's notes and redrive record: the pair it +/// holds is restored from it after a restart. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PrivatePaymentRecord { + /// The account whose notes pay for it. + pub account_index: u32, + /// The spend's activity id, which keys its redrive record. + pub activity_id: [u8; 32], + /// The recorded anchor the spend was built against: once Platform no + /// longer records it, the spend can never execute, which with an input + /// note still unspent and the entry absent from the store proves the + /// payment expired. + pub anchor: [u8; 32], + /// The proved height of the recorded-anchor set `anchor` was chosen from: + /// the anchor was recorded then, so only its absence from a set proved at + /// or above this height shows it pruned. `u64::MAX` for a payment saved + /// before the journal kept it, whose expiry is never proven. + pub anchor_height: u64, + /// The commitment of the entry it appends, which keys the record. + pub commitment: [u8; 32], + /// The request it pays for. + pub payment: PrivateContactPayment, + /// The nullifiers of the notes its spend consumes; never empty. + pub nullifiers: Vec<[u8; 32]>, +} + +impl PrivatePaymentRecord { + /// The format version [`Self::to_bytes`] writes. Versions 1 and 2, written + /// before the anchor and its height were kept, do not read back: their + /// wallet's holds stay. + const FORMAT_VERSION: u8 = 3; + /// The bytes before the nullifiers: version, account index, activity id, + /// anchor, anchor height, commitment, sender, recipient, account reference + /// and nullifier count. + const HEADER_LEN: usize = 1 + 4 + 32 + 32 + 8 + 32 + 32 + 32 + 4 + 2; + + /// The bytes the file store keeps it as: [`Self::FORMAT_VERSION`], the + /// account index little-endian, the activity id, the anchor, the anchor + /// height little-endian, the commitment, the sender, the recipient, the + /// account reference little-endian, the nullifier count as a + /// little-endian `u16`, then the nullifiers. `None` when it has no + /// nullifier or more than a `u16` counts. + pub(super) fn to_bytes(&self) -> Option> { + let count = u16::try_from(self.nullifiers.len()).ok()?; + if count == 0 { + return None; + } + let mut bytes = Vec::with_capacity(Self::HEADER_LEN + 32 * self.nullifiers.len()); + bytes.push(Self::FORMAT_VERSION); + bytes.extend_from_slice(&self.account_index.to_le_bytes()); + bytes.extend_from_slice(&self.activity_id); + bytes.extend_from_slice(&self.anchor); + bytes.extend_from_slice(&self.anchor_height.to_le_bytes()); + bytes.extend_from_slice(&self.commitment); + bytes.extend_from_slice(&self.payment.sender); + bytes.extend_from_slice(&self.payment.recipient); + bytes.extend_from_slice(&self.payment.account_reference.to_le_bytes()); + bytes.extend_from_slice(&count.to_le_bytes()); + for nullifier in &self.nullifiers { + bytes.extend_from_slice(nullifier); + } + Some(bytes) + } + + /// The record [`Self::to_bytes`] wrote, or `None` for any other bytes: a + /// different version, a length that does not match the count, or no + /// nullifier. + pub(super) fn from_bytes(bytes: &[u8]) -> Option { + let (&version, rest) = bytes.split_first()?; + if version != Self::FORMAT_VERSION { + return None; + } + let (account_index, rest) = rest.split_first_chunk::<4>()?; + let (activity_id, rest) = rest.split_first_chunk::<32>()?; + let (anchor, rest) = rest.split_first_chunk::<32>()?; + let (anchor_height, rest) = rest.split_first_chunk::<8>()?; + let (commitment, rest) = rest.split_first_chunk::<32>()?; + let (sender, rest) = rest.split_first_chunk::<32>()?; + let (recipient, rest) = rest.split_first_chunk::<32>()?; + let (account_reference, rest) = rest.split_first_chunk::<4>()?; + let (count, rest) = rest.split_first_chunk::<2>()?; + let count = usize::from(u16::from_le_bytes(*count)); + let (nullifiers, remainder) = rest.as_chunks::<32>(); + if count == 0 || nullifiers.len() != count || !remainder.is_empty() { + return None; + } + Some(Self { + account_index: u32::from_le_bytes(*account_index), + activity_id: *activity_id, + anchor: *anchor, + anchor_height: u64::from_le_bytes(*anchor_height), + commitment: *commitment, + payment: PrivateContactPayment { + sender: *sender, + recipient: *recipient, + account_reference: u32::from_le_bytes(*account_reference), + }, + nullifiers: nullifiers.to_vec(), + }) + } +} + /// What the wallet knows about an identity debit's local recovery state. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum IdentityDebitRecoveryStatus { @@ -219,7 +375,10 @@ pub(super) struct MarkSpentOutcome { /// /// All mutating methods take `&mut self` so implementations can /// batch writes without interior mutability. -pub trait ShieldedStore: Send + Sync { +/// +/// A store is `'static`: a spend dropped before it hands its notes over +/// releases their reservation on a spawned task when the store is locked. +pub trait ShieldedStore: Send + Sync + 'static { /// The error type returned by storage operations. type Error: StdError + Send + Sync + 'static; @@ -265,27 +424,66 @@ pub trait ShieldedStore: Send + Sync { /// got removed; `false` is a no-op (paired clear from the /// rollback path on a transition that never marked pending, /// or a stale clear after the spend already promoted to - /// `mark_spent`). + /// `mark_spent`). Clears whoever holds it: for a resolution of the + /// spend (a pruned anchor). A spend attempt releases its own with + /// [`Self::clear_pending_reserved_by`]. fn clear_pending(&mut self, id: SubwalletId, nullifier: &[u8; 32]) -> Result; - /// Attach the recorded `anchor` the spend was built against, and - /// the linked `activity_id`, to `id`'s pending reservation for - /// `nullifier` (taken earlier by [`Self::mark_pending`]). No-op - /// if the nullifier is no longer pending. The sync reconcile uses - /// the stored anchor to detect a stranded spend whose anchor - /// Platform has pruned (see [`Self::stale_pending_spends`]). + /// [`Self::mark_pending`] for the spend attempt `reservation`, which owns + /// the reservation: only [`Self::clear_pending_reserved_by`] with the + /// same attempt, or a resolution of the spend, clears it. + fn mark_pending_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result; + + /// Release `id`'s reservation of `nullifier`, and the redrive records + /// carrying it, only while the spend attempt `reservation` still holds + /// it ([`Self::mark_pending_by`]), compared and cleared in one call + /// under the store lock: a stale attempt's cleanup never releases a + /// newer attempt's notes or deletes its records. Returns whether it was + /// released. + fn clear_pending_reserved_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result; + + /// Attach the recorded `anchor` the spend was built against, the proved + /// height of the anchor set it was read from (`anchor_height`), and the + /// linked `activity_id`, to `id`'s pending reservation for `nullifier` + /// (taken earlier by [`Self::mark_pending`]). No-op if the nullifier is + /// no longer pending. The sync reconcile uses the stored anchor and its + /// height to detect a stranded spend whose anchor Platform has pruned + /// (see [`Self::stale_pending_spends`]). fn set_pending_spend( &mut self, id: SubwalletId, nullifier: &[u8; 32], anchor: [u8; 32], activity_id: [u8; 32], + anchor_height: Option, + ) -> Result<(), Self::Error>; + + /// Note that `anchor` was in Platform's recorded set proved at `height`: + /// each of `id`'s reservations built on it and each of its records that + /// carry no anchor height get `height`, a record's durably. A set proved + /// at or above it that lacks the anchor then shows it pruned. One that + /// already carries a height keeps it. + fn note_anchor_recorded( + &mut self, + id: SubwalletId, + anchor: &[u8; 32], + height: u64, ) -> Result<(), Self::Error>; /// Pending reservations for `id` that carry a recorded anchor — - /// `(nullifier, anchor, activity_id)` per built-but-unconfirmed - /// spend. Empty when `id` has no anchored reservations (the common + /// `(nullifier, anchor, activity_id, anchor_height)` per + /// built-but-unconfirmed spend. Empty when `id` has no anchored reservations (the common /// case, which lets the sync reconcile skip its network round-trip). /// The reconcile checks each anchor against Platform's recorded set /// and releases the reservation via [`Self::clear_pending`] when the @@ -304,6 +502,94 @@ pub trait ShieldedStore: Send + Sync { /// continue across relaunches. fn arm_redrive(&mut self, id: SubwalletId, redrive: PendingRedrive) -> Result<(), Self::Error>; + /// [`Self::arm_redrive`] for a spend paying for the DashPay private + /// contact request `payment`, which is kept with the record and dropped + /// with it, and the private payment journal's record of each entry the + /// spend appends (`commitments`), which outlives it + /// ([`PrivatePaymentRecord`]). All of it is saved in one write that is + /// durable once this returns, across a crash or power loss too: the + /// spend is broadcast only after it. A store that cannot keep all of it + /// durably fails, and the payment is refused. + /// `anchor_height` is the proved height of the recorded-anchor set the + /// spend's anchor was chosen from ([`PrivatePaymentRecord::anchor_height`]). + fn arm_private_contact_payment( + &mut self, + id: SubwalletId, + redrive: PendingRedrive, + payment: PrivateContactPayment, + commitments: &[[u8; 32]], + anchor_height: u64, + ) -> Result<(), Self::Error>; + + /// The armed records of `wallet_id` that pay for a private contact + /// request, each with the request ([`Self::arm_private_contact_payment`]). + /// Fails when one of them carries an association that does not read. + fn private_contact_redrives_for_wallet( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error>; + + // ── Private payment journal (per-wallet) ─────────────────────────── + + /// Whether `wallet_id`'s private payment journal is open + /// ([`Self::open_private_payment_journal`]): then it is authoritative for + /// the wallet's private payments, so one marked unresolved with no record + /// in it has resolved or was never broadcast. A journal not open yet, as + /// in a store that was deleted or replaced, cannot tell. + fn private_payment_journal_open(&self, wallet_id: WalletId) -> Result; + + /// Open `wallet_id`'s private payment journal, durably: called once a + /// restoration of the wallet's private payments from it succeeded, before + /// the wallet can pay. Saving a journal record opens it too. + fn open_private_payment_journal(&mut self, wallet_id: WalletId) -> Result<(), Self::Error>; + + /// Every journal record of `wallet_id`'s private payments. Fails when one + /// of the wallet's records, or one whose wallet does not read, does not + /// read back. + fn private_payment_journal( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error>; + + /// Add `records` to `wallet_id`'s journal durably, for payments whose + /// redrive record was saved before the journal kept them. + fn journal_private_payments( + &mut self, + wallet_id: WalletId, + records: &[PrivatePaymentRecord], + ) -> Result<(), Self::Error>; + + /// Remove `wallet_id`'s journal record of the entry `commitment`, once + /// its payment provably resolved: the entry was read back, or the payment + /// is proven expired (its anchor pruned and its entry absent from the + /// store). Returns whether there was one. Only this call and + /// [`Self::retire_undelivered_spend`] remove a journal record: not a note + /// release, a redrive's removal or a purge. + fn discharge_private_payment( + &mut self, + wallet_id: WalletId, + commitment: &[u8; 32], + ) -> Result; + + /// Retire the saved record of `id`'s spend `activity_id`, whose + /// transition never left the wallet, together with the journal records of + /// `commitments` in `id`'s wallet, the entries it would have appended. One + /// durable write removes them all (the file store commits it on its + /// recovery connection, at `synchronous=FULL`), so no restart finds the + /// journal discharged while the signed transition is still saved, or the + /// other way round. + /// + /// Returns false, changing nothing, when no such record is saved, or when + /// a re-broadcast of it was claimed ([`PendingRedrive::attempts`] above + /// 0): the transition may then have left the wallet. The notes the spend + /// reserved are the caller's to release. + fn retire_undelivered_spend( + &mut self, + id: SubwalletId, + activity_id: &[u8; 32], + commitments: &[[u8; 32]], + ) -> Result; + /// Every armed redrive record for `id`. fn pending_redrives(&self, id: SubwalletId) -> Result, Self::Error>; @@ -550,6 +836,14 @@ pub(super) struct PendingSpend { /// The linked activity-log entry id, so a released reservation can /// flip its "Pending" row to "Failed" instead of stranding it. pub activity_id: Option<[u8; 32]>, + /// The spend attempt that reserved the note + /// ([`ShieldedStore::mark_pending_by`]), whose release clears it only + /// while it still holds it. `None` for a reservation restored from a + /// redrive record, which only a resolution of its spend clears. + pub reservation: Option, + /// A proved height at which `anchor` was in Platform's recorded set + /// ([`PendingRedrive::anchor_height`]), when known. + pub anchor_height: Option, } /// Per-subwallet note + sync state used by both the in-memory and @@ -694,6 +988,42 @@ impl SubwalletState { } } + /// [`Self::mark_pending`] for the spend attempt `reservation`, which owns + /// the reservation from now on. + pub(super) fn mark_pending_by(&mut self, nullifier: &[u8; 32], reservation: u64) -> bool { + match self.pending_nullifiers.entry(*nullifier) { + std::collections::btree_map::Entry::Vacant(e) => { + e.insert(PendingSpend { + reservation: Some(reservation), + ..PendingSpend::default() + }); + true + } + std::collections::btree_map::Entry::Occupied(_) => false, + } + } + + /// Release the reservation of `nullifier` only while the spend attempt + /// `reservation` holds it ([`Self::mark_pending_by`]), with the redrive + /// records it carries, as [`Self::clear_pending`] does. Returns whether + /// it was released: another attempt's reservation, or one restored from + /// a redrive record, stays. + pub(super) fn clear_pending_reserved_by( + &mut self, + nullifier: &[u8; 32], + reservation: u64, + ) -> bool { + if self + .pending_nullifiers + .get(nullifier) + .is_some_and(|pending| pending.reservation == Some(reservation)) + { + self.clear_pending(nullifier) + } else { + false + } + } + /// Attach the built spend's recorded `anchor` and linked /// `activity_id` to an existing reservation for `nullifier`. /// No-op if the nullifier is no longer pending (e.g. the @@ -703,11 +1033,43 @@ impl SubwalletState { nullifier: &[u8; 32], anchor: [u8; 32], activity_id: [u8; 32], + anchor_height: Option, ) { if let Some(pending) = self.pending_nullifiers.get_mut(nullifier) { pending.anchor = Some(anchor); pending.activity_id = Some(activity_id); + pending.anchor_height = anchor_height; + } + } + + /// Give `height` to each reservation built on `anchor`, and each record + /// on it, that carries no anchor height + /// ([`ShieldedStore::note_anchor_recorded`]). Returns the activity ids of + /// the records given it. + pub(super) fn note_anchor_recorded(&mut self, anchor: &[u8; 32], height: u64) -> Vec<[u8; 32]> { + for pending in self.pending_nullifiers.values_mut() { + if pending.anchor == Some(*anchor) && pending.anchor_height.is_none() { + pending.anchor_height = Some(height); + } + } + let mut noted = Vec::new(); + for redrive in self.redrives.values_mut() { + if redrive.anchor == *anchor && redrive.anchor_height.is_none() { + redrive.anchor_height = Some(height); + noted.push(redrive.activity_id); + } } + noted + } + + /// The activity ids of the records built on `anchor` that carry no anchor + /// height. + pub(super) fn redrives_without_anchor_height(&self, anchor: &[u8; 32]) -> Vec<[u8; 32]> { + self.redrives + .values() + .filter(|redrive| redrive.anchor == *anchor && redrive.anchor_height.is_none()) + .map(|redrive| redrive.activity_id) + .collect() } /// Reservations that carry a recorded anchor — i.e. built, @@ -722,7 +1084,7 @@ impl SubwalletState { .filter_map(|(nullifier, spend)| { spend .anchor - .map(|anchor| (*nullifier, anchor, spend.activity_id)) + .map(|anchor| (*nullifier, anchor, spend.activity_id, spend.anchor_height)) }) .collect() } @@ -856,6 +1218,13 @@ pub struct InMemoryShieldedStore { checkpoints: Vec, /// Placeholder anchor; production stores compute the real Sinsemilla root. anchor: [u8; 32], + /// The private contact request each armed record pays for, by subwallet + /// and activity id, read joined with the live records. + private_contact_payments: BTreeMap<(SubwalletId, [u8; 32]), PrivateContactPayment>, + /// The private payment journal's records, by wallet and commitment. + private_payment_journal: BTreeMap<(WalletId, [u8; 32]), PrivatePaymentRecord>, + /// The wallets whose private payment journal is open. + private_payment_journals: BTreeSet, } impl InMemoryShieldedStore { @@ -927,15 +1296,54 @@ impl ShieldedStore for InMemoryShieldedStore { .unwrap_or(false)) } + fn mark_pending_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result { + Ok(self + .subwallets + .entry(id) + .or_default() + .mark_pending_by(nullifier, reservation)) + } + + fn clear_pending_reserved_by( + &mut self, + id: SubwalletId, + nullifier: &[u8; 32], + reservation: u64, + ) -> Result { + let released = self + .subwallets + .get_mut(&id) + .is_some_and(|sw| sw.clear_pending_reserved_by(nullifier, reservation)); + Ok(released) + } + fn set_pending_spend( &mut self, id: SubwalletId, nullifier: &[u8; 32], anchor: [u8; 32], activity_id: [u8; 32], + anchor_height: Option, ) -> Result<(), Self::Error> { if let Some(sw) = self.subwallets.get_mut(&id) { - sw.set_pending_spend(nullifier, anchor, activity_id); + sw.set_pending_spend(nullifier, anchor, activity_id, anchor_height); + } + Ok(()) + } + + fn note_anchor_recorded( + &mut self, + id: SubwalletId, + anchor: &[u8; 32], + height: u64, + ) -> Result<(), Self::Error> { + if let Some(sw) = self.subwallets.get_mut(&id) { + sw.note_anchor_recorded(anchor, height); } Ok(()) } @@ -949,10 +1357,136 @@ impl ShieldedStore for InMemoryShieldedStore { } fn arm_redrive(&mut self, id: SubwalletId, redrive: PendingRedrive) -> Result<(), Self::Error> { + self.private_contact_payments + .remove(&(id, redrive.activity_id)); + self.subwallets.entry(id).or_default().arm_redrive(redrive); + Ok(()) + } + + fn arm_private_contact_payment( + &mut self, + id: SubwalletId, + redrive: PendingRedrive, + payment: PrivateContactPayment, + commitments: &[[u8; 32]], + anchor_height: u64, + ) -> Result<(), Self::Error> { + if redrive.nullifiers.is_empty() { + return Err(InMemoryStoreError( + "a private payment's record spends no note".to_string(), + )); + } + if !commitments.is_empty() { + self.private_payment_journals.insert(id.wallet_id); + } + for commitment in commitments { + self.private_payment_journal.insert( + (id.wallet_id, *commitment), + PrivatePaymentRecord { + account_index: id.account_index, + activity_id: redrive.activity_id, + anchor: redrive.anchor, + anchor_height, + commitment: *commitment, + payment, + nullifiers: redrive.nullifiers.clone(), + }, + ); + } + self.private_contact_payments + .insert((id, redrive.activity_id), payment); self.subwallets.entry(id).or_default().arm_redrive(redrive); Ok(()) } + fn private_contact_redrives_for_wallet( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error> { + Ok(self + .pending_redrives_for_wallet(wallet_id)? + .into_iter() + .filter_map(|(id, redrive)| { + self.private_contact_payments + .get(&(id, redrive.activity_id)) + .map(|payment| (redrive, *payment)) + }) + .collect()) + } + + fn private_payment_journal_open(&self, wallet_id: WalletId) -> Result { + Ok(self.private_payment_journals.contains(&wallet_id)) + } + + fn open_private_payment_journal(&mut self, wallet_id: WalletId) -> Result<(), Self::Error> { + self.private_payment_journals.insert(wallet_id); + Ok(()) + } + + fn private_payment_journal( + &self, + wallet_id: WalletId, + ) -> Result, Self::Error> { + Ok(self + .private_payment_journal + .range((wallet_id, [0u8; 32])..=(wallet_id, [u8::MAX; 32])) + .map(|(_, record)| record.clone()) + .collect()) + } + + fn journal_private_payments( + &mut self, + wallet_id: WalletId, + records: &[PrivatePaymentRecord], + ) -> Result<(), Self::Error> { + if records.iter().any(|record| record.nullifiers.is_empty()) { + return Err(InMemoryStoreError( + "a private payment's record spends no note".to_string(), + )); + } + if !records.is_empty() { + self.private_payment_journals.insert(wallet_id); + } + for record in records { + self.private_payment_journal + .insert((wallet_id, record.commitment), record.clone()); + } + Ok(()) + } + + fn discharge_private_payment( + &mut self, + wallet_id: WalletId, + commitment: &[u8; 32], + ) -> Result { + Ok(self + .private_payment_journal + .remove(&(wallet_id, *commitment)) + .is_some()) + } + + fn retire_undelivered_spend( + &mut self, + id: SubwalletId, + activity_id: &[u8; 32], + commitments: &[[u8; 32]], + ) -> Result { + let Some(subwallet) = self + .subwallets + .get_mut(&id) + .filter(|subwallet| subwallet.redrive_attempts(activity_id) == Some(0)) + else { + return Ok(false); + }; + subwallet.clear_redrive(activity_id); + self.private_contact_payments.remove(&(id, *activity_id)); + for commitment in commitments { + self.private_payment_journal + .remove(&(id.wallet_id, *commitment)); + } + Ok(true) + } + fn pending_redrives(&self, id: SubwalletId) -> Result, Self::Error> { Ok(self .subwallets @@ -1115,7 +1649,11 @@ impl ShieldedStore for InMemoryShieldedStore { } fn purge_wallet(&mut self, wallet_id: WalletId) -> Result<(), Self::Error> { + // The private payment journal outlives the purge: the payments it + // keeps may still execute. self.subwallets.retain(|id, _| id.wallet_id != wallet_id); + self.private_contact_payments + .retain(|(id, _), _| id.wallet_id != wallet_id); Ok(()) } @@ -1137,6 +1675,7 @@ impl ShieldedStore for InMemoryShieldedStore { fn purge_all_subwallets(&mut self) -> Result<(), Self::Error> { self.subwallets.clear(); + self.private_contact_payments.clear(); Ok(()) } @@ -1372,6 +1911,7 @@ mod tests { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; // Landing path: mark_spent on one nullifier drops the record. @@ -1553,18 +2093,18 @@ mod tests { "newly reserved" ); store - .set_pending_spend(id, &nullifier, anchor, activity_id) + .set_pending_spend(id, &nullifier, anchor, activity_id, Some(10)) .unwrap(); assert_eq!( store.stale_pending_spends(id).unwrap(), - vec![(nullifier, anchor, Some(activity_id))], + vec![(nullifier, anchor, Some(activity_id), Some(10))], "an armed reservation surfaces as an anchored (stale-checkable) spend" ); // Anchor IS still recorded → retain. let recorded: HashSet<[u8; 32]> = [anchor].into_iter().collect(); - for (n, a, _) in store.stale_pending_spends(id).unwrap() { + for (n, a, _, _) in store.stale_pending_spends(id).unwrap() { if !recorded.contains(&a) { store.clear_pending(id, &n).unwrap(); } @@ -1598,13 +2138,13 @@ mod tests { .unwrap(); store.mark_pending(id, &nullifier).unwrap(); store - .set_pending_spend(id, &nullifier, anchor, activity_id) + .set_pending_spend(id, &nullifier, anchor, activity_id, Some(10)) .unwrap(); assert!(store.get_unspent_notes(id).unwrap().is_empty()); // Anchor is ABSENT from the recorded set (pruned) → release. let recorded: HashSet<[u8; 32]> = [[0xEE; 32]].into_iter().collect(); - for (n, a, _) in store.stale_pending_spends(id).unwrap() { + for (n, a, _, _) in store.stale_pending_spends(id).unwrap() { if !recorded.contains(&a) { assert!(store.clear_pending(id, &n).unwrap()); } @@ -1677,6 +2217,7 @@ mod tests { attempts: 7, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; let note_spend = PendingRedrive { activity_id: [0x44; 32], @@ -1686,6 +2227,7 @@ mod tests { attempts: 1, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }; store.arm_redrive(id, identity_guard.clone()).unwrap(); store.arm_redrive(id, note_spend).unwrap(); diff --git a/packages/rs-platform-wallet/src/wallet/shielded/sync.rs b/packages/rs-platform-wallet/src/wallet/shielded/sync.rs index 7d4cb149951..1fbbc4e4418 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/sync.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/sync.rs @@ -153,6 +153,15 @@ pub struct MultiSyncNotesResult { /// subwallet. The caller decides whether to queue it on the /// shared `WalletPersister`. pub changeset: ShieldedChangeSet, + /// The highest proved height this pass's scan covers: every action + /// appended at or below it, and so every nullifier spent there, was + /// scanned, by this pass or an earlier one, and replayed against the + /// notes. It is the height of a batch in the run scanned without a gap + /// from the pass's start whose proof, and every proof read before it, + /// shows the commitment tree ending within that run (the batch's + /// `total_count` is the highest seen). `None` when no batch shows that: + /// a proof of a longer tree than the run reached. + pub covered_height: Option, } impl MultiSyncNotesResult { @@ -471,6 +480,12 @@ where // The stream lives in this block so it — and the up-to-`max_concurrent` // network fetches it holds in flight — is dropped before the finalize // below runs, whether the loop ran to the end or was cancelled. + // What this pass's scan covers (`MultiSyncNotesResult::covered_height`): + // the end of the run of positions scanned without a gap from + // `aligned_start`, and the highest proved height of a batch in it whose + // proof shows the tree ending within it. + let mut scanned_to = aligned_start; + let mut covered_height: Option = None; let cancelled = { let stream = make_stream(aligned_start, tree_size); futures::pin_mut!(stream); @@ -501,6 +516,13 @@ where // it. Stays 0 (indeterminate) only if no batch is ever produced. total_target = total_target.max(batch.total_count); total_notes_scanned += batch.notes.len() as u64; + if batch.start_index <= scanned_to { + let end = batch.start_index.saturating_add(batch.notes.len() as u64); + scanned_to = scanned_to.max(end); + if end >= batch.total_count { + covered_height = covered_height.max(Some(batch.block_height)); + } + } if !batch.notes.is_empty() { last_nonempty = Some((batch.start_index, batch.is_partial)); } @@ -839,6 +861,7 @@ where per_subwallet_newly_spent: newly_spent_per_subwallet, total_scanned: scanned_volume, changeset, + covered_height, }) } diff --git a/packages/rs-platform-wallet/src/wallet/shielded/sync/cancel_tests.rs b/packages/rs-platform-wallet/src/wallet/shielded/sync/cancel_tests.rs index a683da2d53c..8c89aeb5c1a 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/sync/cancel_tests.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/sync/cancel_tests.rs @@ -1,4 +1,5 @@ -//! Cancellation of a note-sync pass (dashpay/platform#5200). +//! Cancellation of a note-sync pass (dashpay/platform#5200), and the +//! height a completed pass covers. //! //! Feeds `sync_notes_from_stream` a synthetic batch stream so the //! cancel path runs without a network. The stream yields real-shaped @@ -351,3 +352,61 @@ async fn should_return_while_waiting_for_the_store_lock_when_cancelled() { let _ = std::fs::remove_file(&path); } + +/// A batch of `len` notes at `start_index`, proved at `block_height`, whose +/// proof (with every one read before it) shows `total_count` notes. +fn proved_batch( + start_index: u64, + len: u64, + total_count: u64, + block_height: u64, +) -> ShieldedChunkBatch { + ShieldedChunkBatch { + block_height, + ..batch(start_index, len, total_count) + } +} + +/// A completed pass covers the proved height of the last batch of its run +/// whose proof shows the tree ending within that run, and no height when +/// every proof shows a longer tree than the run reached. +#[tokio::test] +async fn should_report_the_height_a_completed_scan_covers() { + let cases: [(&str, Vec, Option); 4] = [ + ( + "the last batch ends the tree", + vec![proved_batch(0, 4, 6, 1_000), proved_batch(4, 2, 6, 1_004)], + Some(1_004), + ), + ( + "a proof of a longer tree than the run reached", + vec![proved_batch(0, 4, 9, 1_000), proved_batch(4, 2, 9, 1_004)], + None, + ), + ( + "an empty chunk past a full one, at a later height", + vec![proved_batch(0, 4, 4, 1_000), proved_batch(4, 0, 4, 1_005)], + Some(1_005), + ), + ( + "a batch past a gap in the run", + vec![proved_batch(0, 4, 4, 1_000), proved_batch(8, 0, 4, 1_009)], + Some(1_000), + ), + ]; + for (case, batches, covered) in cases { + let path = temp_tree_path("covered"); + let store = Arc::new(RwLock::new( + FileBackedShieldedStore::open_path(&path, 100).expect("open store"), + )); + let result = run( + &store, + &one_subwallet(), + &CancellationToken::new(), + futures::stream::iter(batches.into_iter().map(Ok)), + ) + .await; + assert_eq!(result.covered_height, covered, "{case}"); + let _ = std::fs::remove_file(&path); + } +} diff --git a/packages/rs-platform-wallet/src/wallet/shielded/viewing_key_bind_tests.rs b/packages/rs-platform-wallet/src/wallet/shielded/viewing_key_bind_tests.rs index fb1d107e6fa..dd0c190a281 100644 --- a/packages/rs-platform-wallet/src/wallet/shielded/viewing_key_bind_tests.rs +++ b/packages/rs-platform-wallet/src/wallet/shielded/viewing_key_bind_tests.rs @@ -528,6 +528,7 @@ async fn bind_persists_viewing_keys_and_restart_rebinds_seedlessly() { attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .expect("persist pending reservation"); @@ -1080,6 +1081,7 @@ async fn should_keep_wallet_and_coordinator_keys_when_guarded_registration_is_re attempts: 0, identity_nonce_finalized: false, identity_user_abandoned: false, + anchor_height: None, }, ) .unwrap(); diff --git a/packages/rs-sdk-ffi/src/mnemonic_resolver_core_signer.rs b/packages/rs-sdk-ffi/src/mnemonic_resolver_core_signer.rs index 5418123ad43..67eb8702f24 100644 --- a/packages/rs-sdk-ffi/src/mnemonic_resolver_core_signer.rs +++ b/packages/rs-sdk-ffi/src/mnemonic_resolver_core_signer.rs @@ -60,7 +60,17 @@ use std::ffi::c_void; use std::os::raw::c_char; use async_trait::async_trait; +use dash_sdk::dpp::data_contract::DataContract; +use dash_sdk::dpp::identity::{Identity, KeyID}; +use dash_sdk::dpp::platform_value::BinaryData; +use dash_sdk::dpp::prelude::Identifier; use dash_sdk::dpp::secp256k1; +use dash_sdk::dpp::version::PlatformVersion; +use dash_sdk::platform::dashpay::{ + seal_private_contact_request, PrivateContactRequest, PrivateContactRequestSender, +}; +use dash_sdk::platform::private_document_entries::PrivateDocumentEntries; +use dash_sdk::platform::private_document_scan::{PrivateDocumentScan, PrivateDocumentScanKeys}; use key_wallet::bip32::{ChildNumber, DerivationPath, ExtendedPrivKey, ExtendedPubKey}; use key_wallet::dashcore::ecdsa::{EcdsaError, EcdsaPublicKey, EcdsaSecretKey, EcdsaSignature}; use key_wallet::dip9::{ @@ -70,6 +80,7 @@ use key_wallet::dip9::{ }; use key_wallet::signer::{ExtendedPubKeySigner, Signer, SignerMethod}; use key_wallet::Network; +use platform_encryption::PrivateDocumentTarget; use thiserror::Error; use zeroize::Zeroizing; @@ -166,6 +177,13 @@ pub enum MnemonicResolverSignerError { /// Decoding the generated recoverable signature failed. #[error("ECDSA signature conversion failed: {0}")] SignatureConversion(#[source] EcdsaError), + + /// The SDK refused to seal a private contact request with the derived + /// key: a malformed request, a key the identity does not hold, a + /// signature that is not the signing key's, or a contract without the + /// private type. + #[error("private contact request: {0}")] + PrivateContactRequest(dash_sdk::Error), } /// `key_wallet::signer::Signer` implementation that derives ECDSA @@ -242,36 +260,36 @@ impl MnemonicResolverCoreSigner { } } - /// Resolve the mnemonic from the Swift-side callback, derive the BIP-32 - /// extended private key at `path`, and hand it *by reference* to - /// `extract`, returning whatever `extract` produces. + /// Resolve the mnemonic from the Swift-side callback, build the BIP-32 + /// master `ExtendedPrivKey`, and hand it *by reference* to `use_master`, + /// returning whatever `use_master` produces. An operation that needs keys + /// at several paths derives them all from this one resolve. /// /// This is the single entry-point for all private-key material in this /// signer. It handles the full stack: resolver FFI call → result-code /// mapping → UTF-8 + word-list validation → BIP-39 seed → master - /// `ExtendedPrivKey` → child `ExtendedPrivKey` at `path`. + /// `ExtendedPrivKey`. /// /// # Zeroization contract /// - /// Both the `master` and `derived` extended keys wipe their secret - /// material when they leave this scope — [`ExtendedPrivKey`] zeroizes on - /// `Drop` and is not `Copy`, so each move is a real move that leaves no - /// bitwise duplicate behind. The - /// key never crosses the call boundary — `extract` only borrows it — so it - /// cannot outlive the derivation. `extract` returns public material - /// (`ExtendedPubKey`) or a `Zeroizing` scalar copy; the caller wipes the - /// latter on its own drop. The mnemonic and seed buffers are plain arrays - /// and ride [`Zeroizing`] wrappers for the same guarantee. + /// The `master` extended key wipes its secret material when it leaves + /// this scope, as does every child `use_master` derives from it: + /// [`ExtendedPrivKey`] zeroizes on `Drop` and is not `Copy`, so each move + /// is a real move that leaves no bitwise duplicate behind. The master key + /// never crosses the call boundary (`use_master` only borrows it), so it + /// cannot outlive the operation. `use_master` returns public material, a + /// product of the keys, or a `Zeroizing` scalar copy the caller wipes on + /// its own drop. The mnemonic and seed buffers are plain arrays and ride + /// [`Zeroizing`] wrappers for the same guarantee. /// /// # Errors /// /// Propagates [`MnemonicResolverSignerError`] for every failure mode: - /// null handle, resolver FFI errors, encoding/parse failures, and BIP-32 - /// derivation errors. - fn resolve_and_derive( + /// null handle, resolver FFI errors, encoding/parse failures, BIP-32 + /// derivation errors, and whatever `use_master` returns. + fn resolve_master( &self, - path: &DerivationPath, - extract: impl FnOnce(&ExtendedPrivKey) -> T, + use_master: impl FnOnce(&ExtendedPrivKey) -> Result, ) -> Result { if self.resolver_addr == 0 { return Err(MnemonicResolverSignerError::NullHandle); @@ -331,11 +349,25 @@ impl MnemonicResolverCoreSigner { let master = ExtendedPrivKey::new_master(self.network, seed.as_ref()) .map_err(|e| MnemonicResolverSignerError::DerivationFailed(format!("master: {e}")))?; - let derived = master - .derive_priv(path) - .map_err(|e| MnemonicResolverSignerError::DerivationFailed(format!("path: {e}")))?; - Ok(extract(&derived)) + use_master(&master) + } + + /// Resolve the mnemonic, derive the BIP-32 extended private key at + /// `path`, and hand it *by reference* to `extract`, returning whatever + /// `extract` produces. The derived key self-wipes on `Drop` when this + /// returns; see [`Self::resolve_master`] for the full contract. + fn resolve_and_derive( + &self, + path: &DerivationPath, + extract: impl FnOnce(&ExtendedPrivKey) -> T, + ) -> Result { + self.resolve_master(|master| { + let derived = master + .derive_priv(path) + .map_err(|e| MnemonicResolverSignerError::DerivationFailed(format!("path: {e}")))?; + Ok(extract(&derived)) + }) } /// Resolve the mnemonic and derive the raw 32-byte scalar at `path`. @@ -583,6 +615,127 @@ impl MnemonicResolverCoreSigner { private_data, }) } + + /// Sort `page`, read from `start_position` of the store of the private + /// document type `document_type_name` of `data_contract_id`, with the + /// wallet's keys ([`PrivateDocumentScanKeys::scan_page`]). `recipients` + /// are the identities entries may be sealed to, each with the paths of + /// its keys by key id; `senders` the identities that may have sealed + /// entries, each with the id and path of an encryption key whose + /// outgoing viewing key recovers what it sent. The mnemonic is resolved + /// once and every key is derived from it here; the SDK builds the scan + /// keys from them ([`PrivateDocumentScanKeys::derive`]) and wipes them. + /// Only the sorted entries return. + pub fn scan_private_document_page( + &self, + recipients: &[(Identifier, Vec<(KeyID, DerivationPath)>)], + senders: &[(Identifier, KeyID, DerivationPath)], + data_contract_id: Identifier, + document_type_name: &str, + start_position: u64, + page: &PrivateDocumentEntries, + ) -> Result { + let target = PrivateDocumentTarget { + contract_id: data_contract_id.to_buffer(), + document_type_name, + }; + self.resolve_master(|master| { + // `secret_key_at` hands back a wiping guard; the copy it yields + // is wiped by `derive` once used. + let keys = PrivateDocumentScanKeys::derive(recipients, senders, |path| { + secret_key_at(master, path).map(|secret| secret.0) + })?; + Ok(keys.scan_page(&target, start_position, page)) + }) + } + + /// [`Self::scan_private_document_page`] for several pages of one store, + /// each with the position it was read from: the mnemonic is resolved once + /// for the batch and the scan keys are derived once, then each page is + /// sorted with them, in order. A wallet scanning many pages at once reads + /// the Keychain once per batch, not once per page. + pub fn scan_private_document_pages( + &self, + recipients: &[(Identifier, Vec<(KeyID, DerivationPath)>)], + senders: &[(Identifier, KeyID, DerivationPath)], + data_contract_id: Identifier, + document_type_name: &str, + pages: &[(u64, &PrivateDocumentEntries)], + ) -> Result, MnemonicResolverSignerError> { + let target = PrivateDocumentTarget { + contract_id: data_contract_id.to_buffer(), + document_type_name, + }; + self.resolve_master(|master| { + // As for one page: `derive` wipes each copy `secret_key_at` + // yields once used. + let keys = PrivateDocumentScanKeys::derive(recipients, senders, |path| { + secret_key_at(master, path).map(|secret| secret.0) + })?; + Ok(pages + .iter() + .map(|(start_position, page)| keys.scan_page(&target, *start_position, page)) + .collect()) + }) + } + + /// Seal `request` from `sender`, whose encryption key + /// `sender_encryption_key_id` is the one at `sender_encryption_key_path`, + /// to the key `recipient_key_id` of `recipient`, with `signature`, the + /// signature of the request by the sender's key `signing_key_id` + /// ([`seal_private_contact_request`]). The signature comes from the + /// identity signer, as a transition's does; this signs nothing. + /// `esk_seed` is 32 fresh random bytes, never reused. The encryption key, + /// and the outgoing viewing key the SDK derives from it, stay here; only + /// the sealed entry returns. + #[allow(clippy::too_many_arguments)] + pub fn seal_private_contact_request( + &self, + sender_encryption_key_path: &DerivationPath, + dashpay_contract: &DataContract, + sender: &Identity, + sender_encryption_key_id: KeyID, + signing_key_id: KeyID, + signature: &[u8; 65], + recipient: &Identity, + recipient_key_id: KeyID, + request: &PrivateContactRequest, + esk_seed: &[u8; 32], + platform_version: &PlatformVersion, + ) -> Result { + let secret = + self.resolve_master(|master| secret_key_at(master, sender_encryption_key_path))?; + seal_private_contact_request( + dashpay_contract, + &PrivateContactRequestSender { + identity: sender, + encryption_key_id: sender_encryption_key_id, + encryption_secret_key: &secret.0, + signing_key_id, + signature, + }, + recipient, + recipient_key_id, + request, + esk_seed, + platform_version, + ) + .map_err(MnemonicResolverSignerError::PrivateContactRequest) + } +} + +/// The secp256k1 secret key at `path` below `master`, wiped on drop. The +/// child `ExtendedPrivKey` self-wipes when this returns. +fn secret_key_at( + master: &ExtendedPrivKey, + path: &DerivationPath, +) -> Result { + let derived = master + .derive_priv(path) + .map_err(|e| MnemonicResolverSignerError::DerivationFailed(format!("path: {e}")))?; + let secret = secp256k1::SecretKey::from_secret_bytes(*derived.private_key.to_bytes()) + .map_err(|e| MnemonicResolverSignerError::InvalidScalar(e.to_string()))?; + Ok(WipingSecretKey(secret)) } /// Result of [`MnemonicResolverCoreSigner::contact_info_seal`]. @@ -1226,4 +1379,705 @@ mod tests { assert_eq!(methods.len(), 1); assert!(matches!(methods[0], SignerMethod::Digest)); } + + mod private_contact_requests { + use super::*; + use dash_sdk::dpp::data_contract::accessors::v0::DataContractV0Getters; + use dash_sdk::dpp::identity::accessors::IdentityGettersV0; + use dash_sdk::dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dash_sdk::dpp::identity::identity_public_key::v0::IdentityPublicKeyV0; + use dash_sdk::dpp::identity::{ + IdentityPublicKey, IdentityV0, KeyType, Purpose, SecurityLevel, + }; + use dash_sdk::dpp::system_data_contracts::{load_system_data_contract, SystemDataContract}; + use dash_sdk::dpp::version::PlatformVersion; + use dash_sdk::platform::dashpay::{ + private_contact_request_identities, private_contact_request_outgoing_viewing_key, + private_contact_request_signature, private_contact_request_signing_data, + private_contact_requests_from_scan, ProvedIdentity, + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + }; + use dash_sdk::platform::private_document_entries::PrivateDocumentEntry; + use key_wallet::mnemonic::Mnemonic; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + use key_wallet::wallet::Wallet; + use simple_signer::SingleKeySigner; + use std::collections::BTreeMap; + use std::sync::atomic::{AtomicUsize, Ordering}; + + const SENDER_ID: [u8; 32] = [0xA0; 32]; + const RECIPIENT_ID: [u8; 32] = [0xB0; 32]; + /// Two identities of another wallet, whose entry the scan passes over. + const OUTSIDER_ID: [u8; 32] = [0xC0; 32]; + const CAROL_ID: [u8; 32] = [0xD0; 32]; + /// The authentication key a sender signs its requests with. + const SIGNING_KEY_ID: KeyID = 0; + const SENDER_KEY_ID: KeyID = 1; + const RECIPIENT_KEY_ID: KeyID = 2; + const BLOCK_TIME_MS: u64 = 1_700_000_000_000; + + /// The sender's DashPay encryption key, in the test wallet. + fn sender_key_path() -> DerivationPath { + DerivationPath::from_str("m/9'/1'/5'/0'/0'/0'/1'").expect("valid path") + } + + /// The recipient's DashPay decryption key, in the test wallet. + fn recipient_key_path() -> DerivationPath { + DerivationPath::from_str("m/9'/1'/5'/0'/0'/1'/2'").expect("valid path") + } + + /// The secret key a resident wallet of `ENGLISH_PHRASE` holds at `path`. + fn seed_secret_key(path: &DerivationPath) -> secp256k1::SecretKey { + let seed = Mnemonic::from_phrase(ENGLISH_PHRASE) + .expect("valid mnemonic") + .to_seed(""); + let wallet = Wallet::from_seed_bytes( + *seed, + Network::Testnet, + WalletAccountCreationOptions::None, + ) + .expect("seeded wallet"); + let xprv = wallet + .derive_extended_private_key(path) + .expect("wallet derives the private key at path"); + secp256k1::SecretKey::from_secret_bytes(*xprv.private_key.to_bytes()) + .expect("a valid scalar") + } + + fn fixed_secret_key(byte: u8) -> secp256k1::SecretKey { + secp256k1::SecretKey::from_secret_bytes([byte; 32]).expect("a valid scalar") + } + + /// The sender's signing key, which the identity signer holds, not the + /// resolver. + fn sender_signing_secret_key() -> secp256k1::SecretKey { + fixed_secret_key(0x51) + } + + fn outsider_secret_key() -> secp256k1::SecretKey { + fixed_secret_key(0x41) + } + + fn outsider_signing_secret_key() -> secp256k1::SecretKey { + fixed_secret_key(0x52) + } + + fn key( + id: KeyID, + purpose: Purpose, + security_level: SecurityLevel, + secret_key: &secp256k1::SecretKey, + ) -> IdentityPublicKey { + IdentityPublicKey::V0(IdentityPublicKeyV0 { + id, + purpose, + security_level, + contract_bounds: None, + key_type: KeyType::ECDSA_SECP256K1, + read_only: false, + data: BinaryData::new( + secp256k1::PublicKey::from_secret_key(secret_key) + .serialize() + .to_vec(), + ), + disabled_at: None, + }) + } + + /// A `HIGH` authentication key, which may sign DashPay's documents. + fn signing_key(secret_key: &secp256k1::SecretKey) -> IdentityPublicKey { + key( + SIGNING_KEY_ID, + Purpose::AUTHENTICATION, + SecurityLevel::HIGH, + secret_key, + ) + } + + fn identity(id: [u8; 32], keys: Vec) -> Identity { + Identity::V0(IdentityV0 { + id: Identifier::from(id), + public_keys: keys.into_iter().map(|key| (key.id(), key)).collect(), + balance: 0, + revision: 0, + }) + } + + fn sender() -> Identity { + identity( + SENDER_ID, + vec![ + signing_key(&sender_signing_secret_key()), + key( + SENDER_KEY_ID, + Purpose::ENCRYPTION, + SecurityLevel::MEDIUM, + &seed_secret_key(&sender_key_path()), + ), + ], + ) + } + + fn recipient() -> Identity { + identity( + RECIPIENT_ID, + vec![key( + RECIPIENT_KEY_ID, + Purpose::DECRYPTION, + SecurityLevel::MEDIUM, + &seed_secret_key(&recipient_key_path()), + )], + ) + } + + fn outsider() -> Identity { + identity( + OUTSIDER_ID, + vec![ + signing_key(&outsider_signing_secret_key()), + key( + SENDER_KEY_ID, + Purpose::ENCRYPTION, + SecurityLevel::MEDIUM, + &outsider_secret_key(), + ), + ], + ) + } + + fn carol() -> Identity { + identity( + CAROL_ID, + vec![key( + RECIPIENT_KEY_ID, + Purpose::DECRYPTION, + SecurityLevel::MEDIUM, + &fixed_secret_key(0x42), + )], + ) + } + + fn dashpay() -> DataContract { + load_system_data_contract(SystemDataContract::Dashpay, PlatformVersion::latest()) + .expect("DashPay loads at the latest version") + } + + fn target(contract: &DataContract) -> PrivateDocumentTarget<'static> { + PrivateDocumentTarget { + contract_id: contract.id().to_buffer(), + document_type_name: PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + } + } + + fn request() -> PrivateContactRequest { + PrivateContactRequest { + account_reference: 7, + extended_public_key: [0x33; platform_encryption::COMPACT_XPUB_LEN], + account_label: Some("Alice".to_string()), + auto_accept_proof: None, + shielded_address: Some([0x55; 43]), + } + } + + /// The signature of `request()` from `sender`'s encryption key to the + /// decryption key of `recipient`, as the wallet gets it: an identity + /// signer holding `signing_secret_key` signs the request's signing + /// data with `sender`'s signing key, as it signs a transition. + async fn signature( + contract: &DataContract, + sender: &Identity, + recipient: &Identity, + signing_secret_key: &secp256k1::SecretKey, + ) -> [u8; 65] { + let signing_key = sender + .get_public_key_by_id(SIGNING_KEY_ID) + .expect("the sender has a signing key") + .clone(); + let signer = SingleKeySigner::new_from_slice( + &signing_secret_key.to_secret_bytes(), + Network::Testnet, + ) + .expect("a valid signing key"); + let signing_data = private_contact_request_signing_data( + contract, + sender, + SENDER_KEY_ID, + recipient, + RECIPIENT_KEY_ID, + &request(), + PlatformVersion::latest(), + ) + .expect("the keys check"); + private_contact_request_signature(&signer, &signing_key, &signing_data) + .await + .expect("the identity signer signs") + } + + /// `request` sealed by `sender` with `secret_key`, the encryption key + /// a resident wallet holds, and `signature`, to the decryption key of + /// `recipient`. + fn seed_sealed( + contract: &DataContract, + sender: &Identity, + secret_key: &secp256k1::SecretKey, + signature: &[u8; 65], + recipient: &Identity, + esk_seed: u8, + ) -> BinaryData { + seal_private_contact_request( + contract, + &PrivateContactRequestSender { + identity: sender, + encryption_key_id: SENDER_KEY_ID, + encryption_secret_key: secret_key, + signing_key_id: SIGNING_KEY_ID, + signature, + }, + recipient, + RECIPIENT_KEY_ID, + &request(), + &[esk_seed; 32], + PlatformVersion::latest(), + ) + .expect("the request seals") + } + + /// The test entry: the sender's request to the recipient. + async fn sent_to_recipient(contract: &DataContract, esk_seed: u8) -> BinaryData { + let signature = signature( + contract, + &sender(), + &recipient(), + &sender_signing_secret_key(), + ) + .await; + seed_sealed( + contract, + &sender(), + &seed_secret_key(&sender_key_path()), + &signature, + &recipient(), + esk_seed, + ) + } + + /// Another wallet's entry: the outsider's request to carol. + async fn sent_to_carol(contract: &DataContract, esk_seed: u8) -> BinaryData { + let signature = signature( + contract, + &outsider(), + &carol(), + &outsider_signing_secret_key(), + ) + .await; + seed_sealed( + contract, + &outsider(), + &outsider_secret_key(), + &signature, + &carol(), + esk_seed, + ) + } + + fn page(entries: Vec) -> PrivateDocumentEntries { + PrivateDocumentEntries { + chunk_power: None, + total_count: entries.len() as u64, + entries: entries + .into_iter() + .zip(0u64..) + .map(|(data, position)| PrivateDocumentEntry { + position, + block_height: 10 + position, + block_time_ms: BLOCK_TIME_MS, + data: data.to_vec(), + }) + .collect(), + } + } + + /// The wallet's recipient key paths: the recipient's decryption key. + fn recipient_paths() -> Vec<(Identifier, Vec<(KeyID, DerivationPath)>)> { + vec![( + Identifier::from(RECIPIENT_ID), + vec![(RECIPIENT_KEY_ID, recipient_key_path())], + )] + } + + /// The wallet's sender key paths: the sender's encryption key. + fn sender_paths() -> Vec<(Identifier, KeyID, DerivationPath)> { + vec![( + Identifier::from(SENDER_ID), + SENDER_KEY_ID, + sender_key_path(), + )] + } + + /// The scan keys a resident wallet of the same seed builds. + fn seed_scan_keys() -> PrivateDocumentScanKeys { + PrivateDocumentScanKeys::new() + .with_recipient( + Identifier::from(RECIPIENT_ID), + &[(RECIPIENT_KEY_ID, seed_secret_key(&recipient_key_path()))], + ) + .with_sender( + Identifier::from(SENDER_ID), + SENDER_KEY_ID, + private_contact_request_outgoing_viewing_key(&seed_secret_key( + &sender_key_path(), + )), + ) + } + + unsafe extern "C" fn counting_resolve( + ctx: *const c_void, + wallet_id_bytes: *const u8, + out_buf: *mut c_char, + out_capacity: usize, + out_len: *mut usize, + ) -> i32 { + (*(ctx as *const AtomicUsize)).fetch_add(1, Ordering::SeqCst); + english_resolve(ctx, wallet_id_bytes, out_buf, out_capacity, out_len) + } + + /// The entry the signer seals with the identity signer's signature is + /// byte for byte the one a resident wallet seals with the encryption + /// key at the same path, so a request sent through the Keychain opens + /// and authenticates for every client. + #[tokio::test] + async fn should_seal_the_entry_a_seed_derived_sender_key_seals() { + let contract = dashpay(); + let signature = signature( + &contract, + &sender(), + &recipient(), + &sender_signing_secret_key(), + ) + .await; + let resolver = make_resolver(english_resolve); + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let sealed = signer + .seal_private_contact_request( + &sender_key_path(), + &contract, + &sender(), + SENDER_KEY_ID, + SIGNING_KEY_ID, + &signature, + &recipient(), + RECIPIENT_KEY_ID, + &request(), + &[7; 32], + PlatformVersion::latest(), + ) + .expect("the signer seals the request"); + assert_eq!(sealed, sent_to_recipient(&contract, 7).await); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + /// A page scanned through the signer sorts as with a resident wallet's + /// scan keys: the request to the recipient, and the same request + /// recovered as sent by the sender's outgoing viewing key, while + /// another wallet's entry is passed over. Both read as the sender's + /// signed request, which needs no secret key. + #[tokio::test] + async fn should_scan_a_page_as_seed_derived_scan_keys_do() { + let contract = dashpay(); + let target = target(&contract); + let page = page(vec![ + sent_to_carol(&contract, 1).await, + sent_to_recipient(&contract, 2).await, + ]); + let resolver = make_resolver(english_resolve); + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let scan = signer + .scan_private_document_page( + &recipient_paths(), + &sender_paths(), + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + &page, + ) + .expect("the signer scans the page"); + assert_eq!(scan, seed_scan_keys().scan_page(&target, 0, &page)); + assert_eq!(scan.received.len(), 1, "the request to the recipient"); + assert_eq!(scan.sent.len(), 1, "the request the sender sent"); + assert!(scan.rejected.is_empty()); + assert!(scan.is_complete()); + + let needed = private_contact_request_identities(&scan); + assert_eq!( + needed.keys().copied().collect::>(), + vec![sender().id(), recipient().id()], + "the read checks the sender and the recipient" + ); + // Each identity as a proof at the height the read needs shows it. + let identities: BTreeMap<_, _> = [sender(), recipient()] + .into_iter() + .map(|identity| { + let proved_at_height = needed[&identity.id()]; + ( + identity.id(), + ProvedIdentity { + identity: Some(identity), + proved_at_height, + }, + ) + }) + .collect(); + let read = private_contact_requests_from_scan( + &contract, + scan, + &identities, + PlatformVersion::latest(), + ) + .expect("the requests read"); + assert!(read.rejected.is_empty(), "rejected: {:?}", read.rejected); + assert_eq!(read.received.len(), 1); + assert_eq!(read.received[0].sender_id, sender().id()); + assert_eq!(read.received[0].sender_key_id, SENDER_KEY_ID); + assert_eq!(read.received[0].recipient_key_id, RECIPIENT_KEY_ID); + assert_eq!(read.received[0].request, request()); + assert_eq!(read.sent.len(), 1); + assert_eq!(read.sent[0].sender_key_id, SENDER_KEY_ID); + assert_eq!(read.sent[0].recipient_id, recipient().id()); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + /// Every key a scan needs comes from one Keychain read, not one per + /// key, and a seal reads the Keychain once. + #[tokio::test] + async fn should_resolve_the_mnemonic_once_per_scan_and_once_per_seal() { + let contract = dashpay(); + let page = page(vec![sent_to_recipient(&contract, 3).await]); + let signature = signature( + &contract, + &sender(), + &recipient(), + &sender_signing_secret_key(), + ) + .await; + let resolves = AtomicUsize::new(0); + let resolver = unsafe { + dash_sdk_mnemonic_resolver_create( + &resolves as *const AtomicUsize as *mut c_void, + counting_resolve, + noop_destroy, + ) + }; + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + // Two keys for the recipient and one sender: three derivations. + let recipients = vec![( + Identifier::from(RECIPIENT_ID), + vec![ + (RECIPIENT_KEY_ID, recipient_key_path()), + ( + 3, + DerivationPath::from_str("m/9'/1'/5'/0'/0'/1'/3'").expect("valid path"), + ), + ], + )]; + + signer + .scan_private_document_page( + &recipients, + &sender_paths(), + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + &page, + ) + .expect("the signer scans the page"); + assert_eq!(resolves.load(Ordering::SeqCst), 1); + + signer + .seal_private_contact_request( + &sender_key_path(), + &contract, + &sender(), + SENDER_KEY_ID, + SIGNING_KEY_ID, + &signature, + &recipient(), + RECIPIENT_KEY_ID, + &request(), + &[4; 32], + PlatformVersion::latest(), + ) + .expect("the signer seals the request"); + assert_eq!(resolves.load(Ordering::SeqCst), 2); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + /// A path that is not the sender's encryption key derives a key the + /// SDK refuses to seal with, and the refusal reaches the caller typed. + #[tokio::test] + async fn should_refuse_to_seal_with_a_key_the_sender_does_not_hold() { + let contract = dashpay(); + let signature = signature( + &contract, + &sender(), + &recipient(), + &sender_signing_secret_key(), + ) + .await; + let resolver = make_resolver(english_resolve); + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let err = signer + .seal_private_contact_request( + &recipient_key_path(), + &contract, + &sender(), + SENDER_KEY_ID, + SIGNING_KEY_ID, + &signature, + &recipient(), + RECIPIENT_KEY_ID, + &request(), + &[7; 32], + PlatformVersion::latest(), + ) + .expect_err("the key at that path is not the sender's"); + assert!( + matches!(err, MnemonicResolverSignerError::PrivateContactRequest(_)), + "expected the SDK's refusal, got: {err:?}" + ); + assert!( + err.to_string().contains("secret key"), + "refused for the encryption key, got: {err}" + ); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + /// The signer seals only what the identity signer signed: a + /// signature by a key other than the signing key it names is the + /// SDK's refusal, not an entry. + #[tokio::test] + async fn should_refuse_to_seal_a_signature_by_another_key() { + let contract = dashpay(); + let forged = signature( + &contract, + &sender(), + &recipient(), + &outsider_signing_secret_key(), + ) + .await; + let resolver = make_resolver(english_resolve); + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let err = signer + .seal_private_contact_request( + &sender_key_path(), + &contract, + &sender(), + SENDER_KEY_ID, + SIGNING_KEY_ID, + &forged, + &recipient(), + RECIPIENT_KEY_ID, + &request(), + &[7; 32], + PlatformVersion::latest(), + ) + .expect_err("the signature is not the signing key's"); + assert!( + matches!(err, MnemonicResolverSignerError::PrivateContactRequest(_)), + "expected the SDK's refusal, got: {err:?}" + ); + assert!( + err.to_string().contains("signature"), + "refused for the signature, got: {err}" + ); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + /// A batch of pages scanned through the signer reads the Keychain once + /// and sorts each page as scanning it alone does. + #[tokio::test] + async fn should_resolve_the_mnemonic_once_per_batch_of_pages() { + let contract = dashpay(); + let first = page(vec![ + sent_to_carol(&contract, 5).await, + sent_to_recipient(&contract, 6).await, + ]); + let mut second = page(vec![sent_to_recipient(&contract, 8).await]); + for entry in &mut second.entries { + entry.position += 2; + } + second.total_count = 3; + let resolves = AtomicUsize::new(0); + let resolver = unsafe { + dash_sdk_mnemonic_resolver_create( + &resolves as *const AtomicUsize as *mut c_void, + counting_resolve, + noop_destroy, + ) + }; + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let scans = signer + .scan_private_document_pages( + &recipient_paths(), + &sender_paths(), + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + &[(0, &first), (2, &second)], + ) + .expect("the signer scans the batch"); + assert_eq!(resolves.load(Ordering::SeqCst), 1, "one Keychain read"); + let target = target(&contract); + assert_eq!( + scans, + vec![ + seed_scan_keys().scan_page(&target, 0, &first), + seed_scan_keys().scan_page(&target, 2, &second), + ] + ); + assert_eq!(scans[1].received.len(), 1); + assert_eq!(scans[1].next_position, 3); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + + #[test] + fn should_report_a_missing_mnemonic_from_a_scan() { + let contract = dashpay(); + let resolver = make_resolver(missing_resolve); + let signer = + unsafe { MnemonicResolverCoreSigner::new(resolver, [0u8; 32], Network::Testnet) }; + + let err = signer + .scan_private_document_page( + &recipient_paths(), + &sender_paths(), + contract.id(), + PRIVATE_CONTACT_REQUEST_DOCUMENT_TYPE_NAME, + 0, + &page(Vec::new()), + ) + .expect_err("no mnemonic, no keys"); + assert!( + matches!(err, MnemonicResolverSignerError::NotFound), + "error should be NotFound, got: {err:?}" + ); + + unsafe { dash_sdk_mnemonic_resolver_destroy(resolver) }; + } + } } diff --git a/packages/rs-sdk/src/mock/sdk.rs b/packages/rs-sdk/src/mock/sdk.rs index f4da24120ff..9205c1055b9 100644 --- a/packages/rs-sdk/src/mock/sdk.rs +++ b/packages/rs-sdk/src/mock/sdk.rs @@ -48,6 +48,12 @@ pub struct MockDashPlatformSdk { /// The block time the metadata of a response answered from an expectation reports /// ([Self::set_response_time_ms]); 0 unless set. response_time_ms: u64, + /// The block height the metadata of a response answered from an expectation reports + /// ([Self::set_response_height]); 0 unless set. + response_height: u64, + /// The block height the metadata of a response of one type answered from an expectation + /// reports, by the type's name ([Self::set_response_height_of]), over `response_height`. + response_heights_of: BTreeMap<&'static str, u64>, } impl MockDashPlatformSdk { @@ -80,6 +86,8 @@ impl MockDashPlatformSdk { dapi, sdk: ArcSwapOption::new(None), response_time_ms: 0, + response_height: 0, + response_heights_of: BTreeMap::new(), } } @@ -90,6 +98,22 @@ impl MockDashPlatformSdk { self } + /// Sets the block height the metadata of every response answered from an expectation + /// reports, for code under test that reads the height a response was proved at. + pub fn set_response_height(&mut self, height: u64) -> &mut Self { + self.response_height = height; + self + } + + /// Sets the block height the metadata of every response of type `O` answered from an + /// expectation reports, over [Self::set_response_height]: for code under test that reads + /// two kinds of responses proved at different heights. + pub fn set_response_height_of(&mut self, height: u64) -> &mut Self { + self.response_heights_of + .insert(std::any::type_name::(), height); + self + } + pub(crate) fn set_sdk(&mut self, sdk: Sdk) { self.sdk.store(Some(Arc::new(sdk))); } @@ -608,6 +632,11 @@ impl MockDashPlatformSdk { ResponseMetadata { protocol_version: dpp::version::LATEST_VERSION, time_ms: self.response_time_ms, + height: self + .response_heights_of + .get(std::any::type_name::()) + .copied() + .unwrap_or(self.response_height), ..Default::default() }, Proof::default(), diff --git a/packages/rs-sdk/src/platform/private_document_scan.rs b/packages/rs-sdk/src/platform/private_document_scan.rs index d34ecebd882..c7717488172 100644 --- a/packages/rs-sdk/src/platform/private_document_scan.rs +++ b/packages/rs-sdk/src/platform/private_document_scan.rs @@ -14,8 +14,10 @@ //! decode and validate. `crate::platform::dashpay` does both for DashPay's private contact //! requests. +use std::collections::BTreeMap; use std::future::Future; use std::pin::Pin; +use std::sync::{Arc, Mutex, PoisonError}; use crate::error::StaleNodeError; use crate::platform::private_document_entries::{ @@ -438,6 +440,11 @@ pub struct PrivateDocumentPage { type PageFuture = Pin> + Send>>; type PageStream = Pin> + Send>>; +/// A page read with the height of the proof it was read with. +type HeightedPageFuture = + Pin> + Send>>; +type HeightedPageStream = + Pin> + Send>>; /// Where a walk over a store's pages is. enum PageWalk { @@ -573,6 +580,60 @@ where ) } +/// [`private_document_page_walk`] over `fetch`, which reads a page with the height of its +/// proof, each page yielded with the lowest height of the reads it holds: the read from its +/// start, and those completing it from where an older snapshot ended +/// ([`whole_chunk_pages`]). A start fetched again keeps the height of its last read, the one +/// the walk kept. +fn private_document_page_walk_with_heights( + fetch: F, + start_position: u64, + max_concurrent: usize, +) -> impl Stream> + Send +where + F: Fn(u64) -> HeightedPageFuture + Clone + Send + Sync + 'static, +{ + // The height of the last read from each start, until the page holding + // that start is yielded. Pages hold disjoint ranges of positions, and + // every read a page holds starts inside its range. + let heights: Arc>> = Arc::default(); + let read_heights = Arc::clone(&heights); + let fetch = move |start: u64| -> PageFuture { + let fetch = fetch.clone(); + let heights = Arc::clone(&read_heights); + Box::pin(async move { + let (entries, height) = fetch(start).await?; + heights + .lock() + .unwrap_or_else(PoisonError::into_inner) + .insert(start, height); + Ok(entries) + }) + }; + private_document_page_walk(fetch, start_position, max_concurrent).map(move |page| { + let page = page?; + let start = page.start_position; + let end = start + .saturating_add(page.entries.entries.len() as u64) + .max(start.saturating_add(1)); + let mut heights = heights.lock().unwrap_or_else(PoisonError::into_inner); + let mut read = heights.split_off(&start); + let mut later = read.split_off(&end); + heights.append(&mut later); + drop(heights); + let height = read + .contains_key(&start) + .then(|| read.into_values().min()) + .flatten() + .ok_or_else(|| { + Error::Generic(format!( + "no proved height for the private document page from position {start}" + )) + })?; + Ok((page, height)) + }) +} + /// How many more times a page whose snapshot of the store ends at or before its start is /// fetched before the walk gives up on it ([`fetch_ahead_of`]). const BEHIND_PAGE_REFETCHES: usize = 2; @@ -754,6 +815,64 @@ impl Sdk { }) } + /// [`Self::private_document_pages`], with the height each page's proof was made at, from + /// the response metadata verified with it: for a caller that must show every page, and + /// the store's end the first page reports, at or above a height. A page completed from + /// more than one read, the first from a node behind, carries the lowest of their heights. + /// The pages and the walk are those of [`Self::private_document_pages`]. + pub fn private_document_pages_with_heights( + &self, + data_contract_id: Identifier, + document_type_name: &str, + start_position: u64, + max_concurrent: usize, + ) -> impl Stream> + Send { + let sdk = self.clone(); + let document_type_name = document_type_name.to_string(); + let resolve = { + let sdk = sdk.clone(); + let document_type_name = document_type_name.clone(); + async move { + sdk.fetch_private_document_contract(data_contract_id, &document_type_name) + .await + } + }; + stream::once(resolve).flat_map(move |resolved| -> HeightedPageStream { + match resolved { + Err(error) => Box::pin(stream::once(async move { Err(error) })), + Ok(_) => { + let sdk = sdk.clone(); + let document_type_name = document_type_name.clone(); + let fetch = move |start: u64| -> HeightedPageFuture { + let sdk = sdk.clone(); + let query = PrivateDocumentEntriesQuery::new( + data_contract_id, + document_type_name.clone(), + start, + ); + Box::pin(async move { + let (entries, metadata) = + PrivateDocumentEntries::fetch_with_metadata(&sdk, query, None) + .await?; + let entries = entries.ok_or_else(|| { + Error::Generic(format!( + "no page of the private document type of contract \ + {data_contract_id} from position {start}" + )) + })?; + Ok((entries, metadata.height)) + }) + }; + Box::pin(private_document_page_walk_with_heights( + fetch, + start_position, + max_concurrent, + )) + } + } + }) + } + /// Fetches the proved page of the store of the private type `document_type_name` of /// `data_contract_id` from `start_position`, and sorts its entries with `keys`. The context /// provider is given the contract with the type first when it lacks it @@ -1438,6 +1557,70 @@ mod tests { assert_eq!(positions, (0..30).collect::>()); } + /// A page completed from where an older snapshot ended carries the lower height of its two + /// reads; every other page, the height of its own. + #[tokio::test] + async fn should_yield_a_completed_page_at_the_lowest_height_of_its_reads() { + let fetch = |start: u64| -> HeightedPageFuture { + Box::pin(async move { + // The node answering the page from 12 is behind, at height 7: its store held + // 14 entries. The rest of that chunk, and every other page, come at height 9. + if start == 12 { + return Ok((drive_page(4, 14, start), 7)); + } + Ok((drive_page(4, 30, start), 9)) + }) + }; + let pages: Vec<(PrivateDocumentPage, u64)> = private_document_page_walk_with_heights( + fetch, + 0, + MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES, + ) + .map(|page| page.expect("every page reads")) + .collect() + .await; + let mut positions = Vec::new(); + for (page, height) in &pages { + let expected = if page.start_position == 12 { 7 } else { 9 }; + assert_eq!(*height, expected, "page from {}", page.start_position); + positions.extend(page.entries.entries.iter().map(|entry| entry.position)); + } + assert_eq!(positions, (0..30).collect::>()); + } + + /// A page fetched again, its first read from a node behind that held no entry from its + /// start, carries the height of the read the walk kept. + #[tokio::test] + async fn should_yield_a_page_fetched_again_at_the_height_of_its_kept_read() { + let behind_once = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(true)); + let fetch = move |start: u64| -> HeightedPageFuture { + let behind_once = std::sync::Arc::clone(&behind_once); + Box::pin(async move { + if start == 8 && behind_once.swap(false, std::sync::atomic::Ordering::SeqCst) { + return Ok((drive_page(4, 8, start), 5)); + } + Ok((drive_page(4, 30, start), 9)) + }) + }; + let pages: Vec<(PrivateDocumentPage, u64)> = private_document_page_walk_with_heights( + fetch, + 0, + MAX_CONCURRENT_PRIVATE_DOCUMENT_PAGE_FETCHES, + ) + .map(|page| page.expect("every page reads")) + .collect() + .await; + assert!( + pages.iter().all(|(_, height)| *height == 9), + "{:?}", + pages + .iter() + .map(|(page, height)| (page.start_position, *height)) + .collect::>() + ); + assert_eq!(pages.len(), 8); + } + /// A fetch of Drive's pages of a store of 30 entries in chunks of 4 where the fetch from /// `behind` is answered, its first `times` times, by a node whose store held `behind` /// entries: no entry from there. @@ -1688,6 +1871,59 @@ mod tests { ); } + /// The pages read with their proved heights are those of the plain stream, in its order, + /// each with the height of the response its proof came with. + #[cfg(feature = "mocks")] + #[tokio::test] + async fn should_stream_each_page_with_its_proved_height() { + use crate::platform::private_document_entries::held_contract_provider::HeldContractProvider; + use crate::SdkBuilder; + use dpp::version::PlatformVersion; + + let mut sdk: Sdk = SdkBuilder::new_mock() + .with_version(PlatformVersion::latest()) + .with_context_provider(HeldContractProvider::new( + Some(contract_with_the_private_type()), + false, + )) + .build() + .expect("expected a mock sdk"); + let contract_id = Identifier::from(CONTRACT_ID); + for start in [2, 4, 8, 12] { + sdk.mock() + .expect_fetch( + PrivateDocumentEntriesQuery::new(contract_id, TYPE_NAME, start), + Some(drive_page(4, 14, start)), + ) + .await + .expect("expected to register the page"); + } + let plain: Vec = sdk + .private_document_pages(contract_id, TYPE_NAME, 2, 3) + .map(|page| page.expect("every page reads")) + .collect() + .await; + let with_heights: Vec<(PrivateDocumentPage, u64)> = sdk + .private_document_pages_with_heights(contract_id, TYPE_NAME, 2, 3) + .map(|page| page.expect("every page reads")) + .collect() + .await; + assert_eq!( + with_heights + .iter() + .map(|(page, _)| page.clone()) + .collect::>(), + plain + ); + // The mock answers every page with the same response metadata. + let heights: Vec = with_heights.iter().map(|(_, height)| *height).collect(); + assert_eq!(heights.len(), 4); + assert!( + heights.windows(2).all(|pair| pair[0] == pair[1]), + "{heights:?}" + ); + } + /// DashPay as the latest version has it, under the test contract's id: a contract that /// declares the private type the test entries are sealed for. #[cfg(feature = "mocks")]