Merge #7916: backport: bitcoin#27800, bitcoin#28125, bitcoin#28232, bitcoin#27695

8cfc3c4 Merge bitcoin#27695: test: Add test to check tx in the last block can be downloaded (DCG-Claude)
3e2077c Merge bitcoin#28232: test: locked_wallet, skip default fee estimation (DCG-Claude)
a579bef Merge bitcoin#28125: wallet: bugfix, disallow migration of invalid scripts (DCG-Claude)
2d7b155 Merge bitcoin#27800: streams: Drop confusing DataStream::Serialize method and << operator (DCG-Claude)

Pull request description:

  <!-- backportsys v1 -->
  > 🔎 **Needs a careful look.** Commits are ordered by how much they were adapted: 🔴 heavy first, then 🟡 light (at most two invented lines, or up to six lines counting edits of upstream functional-test or doc lines), then 🟢 faithful (identical to upstream apart from context).

  ## Issue being fixed or feature implemented

  Backports 4 Bitcoin Core v0.26 pull request(s) that the Dash queue selected, including any discovered prerequisites: bitcoin#27800, bitcoin#28125, bitcoin#28232, bitcoin#27695.

  ## What was done?

  | upstream | commit | review | gates | notes |
  |---|---|---|---|---|
  | bitcoin#27800 | `2d7b155784` | 🔴 heavy | build:pass mech:warn pick:pass tests:pass tree:pass verify:pass | Backports bitcoin#27800: removes DataStream::Serialize and updates the upstream call sites in wallet/dump.cpp  |
  | bitcoin#28232 | `3e2077c380` | 🔴 heavy | build:pass mech:warn pick:pass tests:pass tree:pass verify:pass | Backports bitcoin#28232: the locked_wallet test now sets a fixed tx fee so it no longer depends on fee estimat |
  | bitcoin#27695 | `8cfc3c4522` | 🔴 heavy | build:pass ci_upstream:warn mech:warn pick:pass tests:pass tree:pass verify:pass | Backports bitcoin#27695 in full: p2p_leak_tx.py is split into test_tx_in_block and test_notfound_on_unannounce |
  | bitcoin#28125 | `a579befd06` | 🟡 light | build:pass mech:warn pick:pass tests:pass tree:pass verify:pass | Backports bitcoin#28125 in full: legacy-wallet migration now skips invalid or non-watched watch-only scripts ( |

  Each commit keeps the upstream subject (`partial Merge …` only where a hunk is deferred to a prerequisite still to be backported, named in the commit body; a hunk Dash intentionally never wants is recorded in the commit body or the reviewer's note above and does not make the backport partial). Conflicts were resolved commit by commit; commits that needed no resolution were cherry-picked unchanged.

  ## How Has This Been Tested?

  Recorded per commit, at that commit's own sha, not once for the branch:

  - built at each commit, reusing the worktree build cache
  - the test suites touched by each diff run at that commit, where the diff selected one
  - a mechanical diff-of-diffs against the upstream patch (every upstream hunk present, no added line without an upstream counterpart, no Dash-specific line dropped)
  - an independent verification pass on 4 commit(s) that needed adaptation
  - fork gate PR thepastaclaw#100 at `8cfc3c4522` — **no passing CI record at this head**

  Gates that did not come back clean — please weigh these:

  - bitcoin#27800 `mech`: **warn** — 9 invented line(s); 1 Dash-feature line(s) removed; 1 partial/prereq/low-risk finding(s)
  - bitcoin#28125 `mech`: **warn** — 1 edited line(s)
  - bitcoin#28232 `mech`: **warn** — 1 invented line(s); 1 partial/prereq/low-risk finding(s)
  - bitcoin#27695 `mech`: **warn** — 5 invented line(s); 1 partial/prereq/low-risk finding(s)

  ## Breaking Changes

  None beyond the upstream changes themselves.

  ## Checklist:

  _Left for the reviewer; backportsys does not tick boxes on its own behalf._

  - [ ] I have performed a self-review of my own code
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

  ---

  ### Maintainer controls

  <!-- backportsys controls -->
  Tick a box and backportsys acts on it within a few minutes, then clears the box. For anything else — a hunk to drop, a resolution to redo, a question — just leave a review comment; nothing here needs a box.

  - [ ] 🔒 **Hands off** — stop every automated update to this branch <!-- bps:hands_off -->
  - [ ] 🔄 **Rebase** onto current `develop` <!-- bps:rebase -->
  - [ ] ❌ **Close** — abandon this batch and release its items <!-- bps:close -->

  <!-- /backportsys controls -->

  <!-- human notes below; the bot never edits this section -->

ACKs for top commit:
  knst:
    utACK 8cfc3c4

Tree-SHA512: da34d35ae7071be1d25f9bf7e8b06f94e3f81f50c40d1c278230be3af6ee37d74e86417be89409911dffc4b990eb5c35c704ffaa09d51847b6abfc307180be5f
(cherry picked from commit fe232f9)

Author: knst

src/flat-database.h

@@ -59,7 +59,7 @@ class CFlatDB
 
         // Write and commit header, data
         try {
-            fileout << ssObj;
+            fileout << Span{ssObj};
         }
         catch (std::exception &e) {
             LogError("%s: Serialize or I/O error - %s\n", __func__, e.what());

src/governance/net_governance.cpp

@@ -288,15 +288,15 @@ bool NetGovernance::ProcessGetData(CNode& pfrom, const CInv& inv, const CNetMsgM
         CDataStream ss(SER_NETWORK, pfrom.GetCommonVersion());
         ss.reserve(1000);
         if (!m_gov_manager.SerializeObjectForHash(inv.hash, ss)) return false;
-        m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::MNGOVERNANCEOBJECT, ss));
+        m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::MNGOVERNANCEOBJECT, Span{ss}));
         return true;
     }
     if (inv.type == MSG_GOVERNANCE_OBJECT_VOTE) {
         if (!m_gov_manager.HaveVoteForHash(inv.hash)) return false;
         CDataStream ss(SER_NETWORK, pfrom.GetCommonVersion());
         ss.reserve(1000);
         if (!m_gov_manager.SerializeVoteForHash(inv.hash, ss)) return false;
-        m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::MNGOVERNANCEOBJECTVOTE, ss));
+        m_connman.PushMessage(&pfrom, msgMaker.Make(NetMsgType::MNGOVERNANCEOBJECTVOTE, Span{ss}));
         return true;
     }
     return false;

src/llmq/dkgsessionmgr.cpp

@@ -129,7 +129,7 @@ void CDKGSessionManager::WriteVerifiedVvecContribution(Consensus::LLMQType llmqT
     for (auto& pubkey : *vvec) {
         s << CBLSPublicKeyVersionWrapper(pubkey, false);
     }
-    db->Write(std::make_tuple(DB_VVEC, llmqType, pQuorumBaseBlockIndex->GetBlockHash(), proTxHash), s);
+    db->Write(std::make_tuple(DB_VVEC, llmqType, pQuorumBaseBlockIndex->GetBlockHash(), proTxHash), Span{s});
 }
 
 void CDKGSessionManager::WriteVerifiedSkContribution(Consensus::LLMQType llmqType, const CBlockIndex* pQuorumBaseBlockIndex, const uint256& proTxHash, const CBLSSecretKey& skContribution)

src/llmq/net_quorum.cpp

@@ -113,8 +113,8 @@ void NetQuorum::ProcessMessage(CNode& pfrom, const std::string& msg_type, CDataS
             }
             request.SetError(nError);
             CDataStream ssResponse{SER_NETWORK, pfrom.GetCommonVersion()};
-            ssResponse << request << body;
-            m_connman.PushMessage(&pfrom, CNetMsgMaker(pfrom.GetCommonVersion()).Make(NetMsgType::QDATA, ssResponse));
+            ssResponse << request << Span{body};
+            m_connman.PushMessage(&pfrom, CNetMsgMaker(pfrom.GetCommonVersion()).Make(NetMsgType::QDATA, Span{ssResponse}));
             return misbehave;
         };
 

src/llmq/quorums.cpp

@@ -200,7 +200,7 @@ void CQuorum::WriteContributions(CDBWrapper& db) const
         for (auto& pubkey : *quorumVvec) {
             s << CBLSPublicKeyVersionWrapper(pubkey, false);
         }
-        db.Write(std::make_pair(DB_QUORUM_QUORUM_VVEC, dbKey), s);
+        db.Write(std::make_pair(DB_QUORUM_QUORUM_VVEC, dbKey), Span{s});
     }
     if (skShare.IsValid()) {
         db.Write(std::make_pair(DB_QUORUM_SK_SHARE, dbKey), skShare);

src/llmq/quorumsman.cpp

@@ -676,7 +676,7 @@ void CQuorumManager::MigrateOldQuorumDB(CEvoDB& evoDb) const
                 if (!evoDb.GetRawDB().ReadDataStream(k, s)) {
                     break;
                 }
-                batch.Write(k, s);
+                batch.Write(k, Span{s});
             }
             if (prefix == DB_QUORUM_SK_SHARE) {
                 if (!pcursor->GetValue(sk)) {

src/streams.h

@@ -316,14 +316,6 @@ class DataStream
         vch.insert(vch.end(), src.begin(), src.end());
     }
 
-    template<typename Stream>
-    void Serialize(Stream& s) const
-    {
-        // Special case: stream << stream concatenates like stream += stream
-        if (!vch.empty())
-            s.write(MakeByteSpan(vch));
-    }
-
     template<typename T>
     DataStream& operator<<(const T& obj)
     {

src/wallet/dump.cpp

@@ -251,11 +251,7 @@ bool CreateFromDump(const ArgsManager& args, const std::string& name, const fs::
 
             std::vector<unsigned char> k = ParseHex(key);
             std::vector<unsigned char> v = ParseHex(value);
-
-            DataStream ss_key{k};
-            DataStream ss_value{v};
-
-            if (!batch->Write(ss_key, ss_value)) {
+            if (!batch->Write(Span{k}, Span{v})) {
                 error = strprintf(_("Error: Unable to write record to new wallet"));
                 ret = false;
                 break;

src/wallet/scriptpubkeyman.cpp

@@ -1872,8 +1872,23 @@ std::unordered_set<CScript, SaltedSipHasher> LegacyScriptPubKeyMan::GetScriptPub
     }
 
     // All watchonly scripts are raw
-    spks.insert(setWatchOnly.begin(), setWatchOnly.end());
+    for (const CScript& script : setWatchOnly) {
+        // As the legacy wallet allowed to import any script, we need to verify the validity here.
+        // LegacyScriptPubKeyMan::IsMine() return 'ISMINE_NO' for invalid or not watched scripts (IsMineResult::INVALID or IsMineResult::NO).
+        // e.g. a "sh(sh(pkh()))" which legacy wallets allowed to import!.
+        if (IsMine(script) != ISMINE_NO) spks.insert(script);
+    }
+
+    return spks;
+}
 
+std::unordered_set<CScript, SaltedSipHasher> LegacyScriptPubKeyMan::GetNotMineScriptPubKeys() const
+{
+    LOCK(cs_KeyStore);
+    std::unordered_set<CScript, SaltedSipHasher> spks;
+    for (const CScript& script : setWatchOnly) {
+        if (IsMine(script) == ISMINE_NO) spks.insert(script);
+    }
     return spks;
 }
 

src/wallet/scriptpubkeyman.h

@@ -526,6 +526,12 @@ class LegacyScriptPubKeyMan : public ScriptPubKeyMan, public FillableSigningProv
     std::set<CKeyID> GetKeys() const override;
     std::unordered_set<CScript, SaltedSipHasher> GetScriptPubKeys() const override;
 
+    /**
+     * Retrieves scripts that were imported by bugs into the legacy spkm and are
+     * simply invalid, such as a sh(sh(pkh())) script, or not watched.
+     */
+    std::unordered_set<CScript, SaltedSipHasher> GetNotMineScriptPubKeys() const;
+
     /** Get the DescriptorScriptPubKeyMans (with private keys) that have the same scriptPubKeys as this LegacyScriptPubKeyMan.
      * Does not modify this ScriptPubKeyMan. */
     std::optional<MigrationData> MigrateToDescriptor();