From 70741982ad2dfdad1ef02406050354c8929cc1d4 Mon Sep 17 00:00:00 2001 From: Konstantin Akimov Date: Fri, 16 Oct 2026 15:37:33 +0900 Subject: [PATCH 1/4] Merge bitcoin/bitcoin#36439: test: poll more frequently in sync helpers and RPC connection wait MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 8898f832d9d86d5a6a8ad8719e1009c702bbf4bd test: poll more frequently in sync helpers and RPC connection wait (ViniciusCestarii) Pull request description: Tested on my x86-64 machine: - Wall time (-j=18): 110.8s to 96.4s (−13%) - Accumulated: 1715s to 1369s (−20%) Also compared the CI on my fork (time for test_runner.py runtime): - macOS native: [1287s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758264512#step:9:4745) to [1175s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791769239#step:9:4744) (-9%) - 32 bit ARM: [776s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758264778#step:10:6336) to [690s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791769348#step:10:5806) (-11%) - Windows native, VS: [1456s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758264639#step:14:1486) to [1252s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791768933#step:14:1486) (-14%) - previous releases: [930s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758264815#step:10:5510) to [786s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791769195#step:10:6016) (-15%) - No wallet: [614s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758264997#step:10:5421) to [572s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791769524#step:10:5813) (-7%) - Alpine (musl): [1958s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37308619216/job/111758265011#step:10:5041) to [1970s](https://github.com/ViniciusCestarii/bitcoin/actions/runs/37318654530/job/111791769871#step:10:5489) (+0.6%) Sanitizer jobs are left out because they vary too much between runs. Alpine (musl) uses --usecli, so more frequent poll creates a new bitcoin-cli process each time, making it not faster with these changes. Also tried locally with --usecli: from 2060s to 2051s accumulated. ACKs for top commit: maflcko: review ACK 8898f832d9d86d5a6a8ad8719e1009c702bbf4bd 👪 Tree-SHA512: f27f7636c5e6d678f9105189a910a2653fac9b5ca4dfdb132eec70cec5eea6489a8f7b8ced122fe95a080a57f325eb0ba230f6e6a21af3db8a402b117e77619a Co-authored-by: merge-script --- test/functional/test_framework/test_framework.py | 7 ++++--- test/functional/test_framework/test_node.py | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/test/functional/test_framework/test_framework.py b/test/functional/test_framework/test_framework.py index 75125f6a9954..c7c722da7ccf 100755 --- a/test/functional/test_framework/test_framework.py +++ b/test/functional/test_framework/test_framework.py @@ -826,7 +826,7 @@ def generatetodescriptor(self, generator, *args, sync_fun=None, **kwargs): sync_fun() if sync_fun else self.sync_all() return blocks - def sync_blocks(self, nodes=None, wait=1, timeout=60): + def sync_blocks(self, nodes=None, *, wait=0.05, timeout=60): """ Wait until everybody has the same tip. sync_blocks needs to be called with an rpc_connections set that has least @@ -848,7 +848,7 @@ def sync_blocks(self, nodes=None, wait=1, timeout=60): "".join("\n {!r}".format(b) for b in best_hash), )) - def sync_mempools(self, nodes=None, wait=1, timeout=60, flush_scheduler=True, wait_func=None): + def sync_mempools(self, nodes=None, *, wait=0.05, timeout=60, flush_scheduler=True, wait_func=None): """ Wait until everybody has the same transactions in their memory pools @@ -857,7 +857,8 @@ def sync_mempools(self, nodes=None, wait=1, timeout=60, flush_scheduler=True, wa timeout = int(timeout * self.options.timeout_factor) stop_time = time.time() + timeout if self.mocktime != 0 and wait_func is None: - wait_func = lambda: self.bump_mocktime(3, nodes=nodes) + # Polling is much faster than the mocktime bump was designed for; keep the bump at its original cadence + wait_func = throttled(lambda: self.bump_mocktime(3, nodes=nodes)) while time.time() <= stop_time: pool = [set(r.getrawmempool()) for r in rpc_connections] if pool.count(pool[0]) == len(rpc_connections): diff --git a/test/functional/test_framework/test_node.py b/test/functional/test_framework/test_node.py index 7413569ed8cf..90cb83ba782c 100755 --- a/test/functional/test_framework/test_node.py +++ b/test/functional/test_framework/test_node.py @@ -288,9 +288,10 @@ def start(self, extra_args=None, *, cwd=None, stdout=None, stderr=None, env=None def wait_for_rpc_connection(self): """Sets up an RPC connection to the dashd process. Returns False if unable to connect.""" - # Poll at a rate of four times per second - poll_per_s = 4 - for _ in range(poll_per_s * self.rpc_timeout): + # Poll at a rate of twenty times per second + poll_per_s = 20 + stop_time = time.monotonic() + self.rpc_timeout + while time.monotonic() <= stop_time: if self.process.poll() is not None: # Attach abrupt shutdown error/s to the exception message self.stderr.seek(0) From 157cc162c4fa843f5b1244bd0e7e13389dcfc4df Mon Sep 17 00:00:00 2001 From: Konstantin Akimov Date: Tue, 13 Oct 2026 09:36:27 +0900 Subject: [PATCH 2/4] partial Merge bitcoin/bitcoin#36440: test: speed up functional tests by removing avoidable waits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BACKPORT NOTE: Missing changes are in wallet_exported_watchonly.py ------ 8dc80cafa2ee5932c9ff499c02d6ab8e3cf2f5ef test: wake p2p wait_until on message delivery instead of polling (ViniciusCestarii) 10cfeb33357c937ddd9f588a35e0f8f33abd6ece test: set noban_tx_relay to speed up slow tx relay tests (ViniciusCestarii) Pull request description: Reduce functional test runtime: - set noban_tx_relay = True for tests that can benefit from it. - wake P2PInterface.wait_until as soon as a message is delivered or the connection state changes, instead of polling every 50ms. Tested on my x86-64 machine: Wall time (-j=18): 110.8s to 99.1s (-11%) Accumulated: 1715s to 1411s (-17.5%) Tests that were most affected: | test | before | after | |---|---|---| | feature_maxuploadtarget | 57.9s | 6.2s | | wallet_conflicts | 49.2s | 7.0s | | p2p_sendheaders | 31.1s | 6.8s | | p2p_compactblocks | 11.1s | 1.7s | | wallet_orphanedreward | 9.3s | 5.2s | | wallet_importprunedfunds | 4.9s | 2.9s | ACKs for top commit: maflcko: review ACK 8dc80cafa2ee5932c9ff499c02d6ab8e3cf2f5ef 🏏 w0xlt: ACK 8dc80cafa2ee5932c9ff499c02d6ab8e3cf2f5ef Tree-SHA512: e01654de189d33957e5c9794c3be0aad08a1628480af11ab2365bd6c8b5d6e1130e561fa85380e2b23daad21248993a5658f1740d9d03d1fe5485eef0a9c3851 Co-authored-by: merge-script --- test/functional/test_framework/p2p.py | 9 ++++++++- test/functional/test_framework/util.py | 6 ++++++ test/functional/wallet_change_address.py | 3 +++ test/functional/wallet_conflicts.py | 2 ++ test/functional/wallet_importprunedfunds.py | 2 ++ test/functional/wallet_orphanedreward.py | 2 ++ test/functional/wallet_signer.py | 4 +++- 7 files changed, 26 insertions(+), 2 deletions(-) diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py index 7601e6eba5f2..9a257f31a673 100755 --- a/test/functional/test_framework/p2p.py +++ b/test/functional/test_framework/p2p.py @@ -294,6 +294,8 @@ def connection_made(self, transport): if self.p2p_connected_to_node and not self.supports_v2_p2p: self.send_version() self.on_open() + with p2p_lock: + p2p_lock.notify_all() def connection_lost(self, exc): """asyncio callback when a connection is closed.""" @@ -305,6 +307,8 @@ def connection_lost(self, exc): self._transport = None self.recvbuf = b"" self.on_close() + with p2p_lock: + p2p_lock.notify_all() # v2 handshake method def _on_data_v2_handshake(self): @@ -572,6 +576,7 @@ def on_message(self, message): except Exception: print("ERROR delivering %s (%s)" % (repr(message), sys.exc_info()[0])) raise + p2p_lock.notify_all() # Callback methods. Can be overridden by subclasses in individual test # cases to provide custom message handling behaviour. @@ -782,7 +787,9 @@ def test_function(): # P2PConnection acquires this lock whenever delivering a message to a P2PInterface. # This lock should be acquired in the thread running the test logic to synchronize # access to any data shared with the P2PInterface or P2PConnection. -p2p_lock = threading.Lock() +# It is a Condition so that waiters can be woken up as soon as a message is +# delivered or the connection state changes, instead of polling. +p2p_lock = threading.Condition(threading.Lock()) class NetworkThread(threading.Thread): diff --git a/test/functional/test_framework/util.py b/test/functional/test_framework/util.py index 812ebcf089cb..da39edee1a57 100644 --- a/test/functional/test_framework/util.py +++ b/test/functional/test_framework/util.py @@ -18,6 +18,7 @@ import shutil import re import sys +import threading import time import urllib.parse @@ -281,6 +282,11 @@ def wait_until_helper(predicate, *, attempts=float('inf'), timeout=float('inf'), with lock: if predicate(): return True + if isinstance(lock, threading.Condition): + # Wake up early when the lock owner signals a state change + lock.wait(sleep) + attempt += 1 + continue else: if predicate(): return True diff --git a/test/functional/wallet_change_address.py b/test/functional/wallet_change_address.py index f71ed1b2e809..40c5bfa1c45d 100755 --- a/test/functional/wallet_change_address.py +++ b/test/functional/wallet_change_address.py @@ -26,6 +26,9 @@ def set_test_params(self): ["-discardfee=1"], ["-avoidpartialspends", "-discardfee=1"] ] + # whitelist peers to speed up tx relay / mempool sync + for args in self.extra_args: + args.append("-whitelist=noban@127.0.0.1") def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/wallet_conflicts.py b/test/functional/wallet_conflicts.py index 802b718cd5e2..4a5efb08728e 100755 --- a/test/functional/wallet_conflicts.py +++ b/test/functional/wallet_conflicts.py @@ -20,6 +20,8 @@ def add_options(self, parser): def set_test_params(self): self.num_nodes = 3 + # whitelist peers to speed up tx relay / mempool sync + self.extra_args = [["-whitelist=noban@127.0.0.1"]] * self.num_nodes def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/wallet_importprunedfunds.py b/test/functional/wallet_importprunedfunds.py index 42b74e5c7e1a..984ec467bba7 100755 --- a/test/functional/wallet_importprunedfunds.py +++ b/test/functional/wallet_importprunedfunds.py @@ -27,6 +27,8 @@ def add_options(self, parser): def set_test_params(self): self.setup_clean_chain = True self.num_nodes = 2 + # whitelist peers to speed up tx relay / mempool sync + self.extra_args = [["-whitelist=noban@127.0.0.1"]] * self.num_nodes def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/wallet_orphanedreward.py b/test/functional/wallet_orphanedreward.py index c96d67d84451..307ffa4a11ce 100755 --- a/test/functional/wallet_orphanedreward.py +++ b/test/functional/wallet_orphanedreward.py @@ -15,6 +15,8 @@ def add_options(self, parser): def set_test_params(self): self.setup_clean_chain = True self.num_nodes = 2 + # whitelist peers to speed up tx relay / mempool sync + self.extra_args = [["-whitelist=noban@127.0.0.1"]] * self.num_nodes def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/wallet_signer.py b/test/functional/wallet_signer.py index d15d217749ec..907127eb650c 100755 --- a/test/functional/wallet_signer.py +++ b/test/functional/wallet_signer.py @@ -44,11 +44,13 @@ def mock_multi_signers_path(self): def set_test_params(self): self.num_nodes = 2 - self.extra_args = [ [], [f"-signer={self.mock_signer_path()}", '-keypool=10'], ] + # whitelist peers to speed up tx relay / mempool sync + for args in self.extra_args: + args.append("-whitelist=noban@127.0.0.1") def skip_test_if_missing_module(self): self.skip_if_no_external_signer() From f9141f5703e343c03d36da7fc60f526b1cad1eb1 Mon Sep 17 00:00:00 2001 From: Konstantin Akimov Date: Mon, 5 Oct 2026 09:27:49 +0200 Subject: [PATCH 3/4] Merge bitcoin/bitcoin#36327: test: speed up secp256k1 fixed-base multiplication and Schnorr signing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit b6fc3cf0046d3106e0dfb30f320d48514eff6fb2 test: speed up secp256k1 fixed-base multiplication (brunoerg) Pull request description: This PR speeds up the test framework’s fixed-base multiplication and Schnorr signing by: - Changing FastGEMul to process scalars in 4-bit windows instead of individual bits. Precompute all 16 multiples for each window position, reducing multiplication from about 128 point additions on average to 64 table lookups and about 60 nonzero point additions. This trades a larger table and additional initialization work for cheaper repeated multiplications. On my local machine (Macbook M2 Pro) the `feature_taproot.py`'s medium time decreased from 83.39s to 35.42s. ACKs for top commit: fametrano: tACK b6fc3cf0046d3106e0dfb30f320d48514eff6fb2 theStack: ACK b6fc3cf0046d3106e0dfb30f320d48514eff6fb2 ViniciusCestarii: tACK b6fc3cf0046d3106e0dfb30f320d48514eff6fb2 Tree-SHA512: 983c1e971122e7a15458436943d5c14a936e86b157bc8c74ce1631aafcd0c7f35bbc476df625a57afb37d4e32bb620d1f1d5b3e2267867279bca736f170e83c9 Co-authored-by: merge-script --- .../test_framework/crypto/secp256k1.py | 39 ++++++++++++------- 1 file changed, 26 insertions(+), 13 deletions(-) diff --git a/test/functional/test_framework/crypto/secp256k1.py b/test/functional/test_framework/crypto/secp256k1.py index 2e9e419da5b0..225ed2dc70f5 100644 --- a/test/functional/test_framework/crypto/secp256k1.py +++ b/test/functional/test_framework/crypto/secp256k1.py @@ -15,6 +15,7 @@ * G: the secp256k1 generator point """ +import functools class FE: """Objects of this class represent elements of the field GF(2**256 - 2**32 - 977). @@ -227,7 +228,7 @@ def mul(*aps): def __rmul__(self, a): """Multiply an integer with a group element.""" if self == G: - return FAST_G.mul(a) + return mul_G(a % GE.ORDER) return GE.mul((a, self)) def __neg__(self): @@ -319,28 +320,40 @@ def __repr__(self): class FastGEMul: """Table for fast multiplication with a constant group element. - Speed up scalar multiplication with a fixed point P by using a precomputed lookup table with - its powers of 2: + Speed up scalar multiplication with a fixed point P by using a precomputed lookup table. + The scalar is split into WINDOW-bit chunks, and for every chunk position i the table holds + a row [(0 * 2^(WINDOW*i)) * P, (1 * 2^(WINDOW*i)) * P, ..., ((2^WINDOW-1) * 2^(WINDOW*i)) * P]: - table = [P, 2*P, 4*P, (2^3)*P, (2^4)*P, ..., (2^255)*P] + table[i][j] = j * (2^(WINDOW*i)) * P - During multiplication, the points corresponding to each bit set in the scalar are added up, - i.e. on average ~128 point additions take place. + During multiplication, one table entry per chunk is added up, i.e. 256/WINDOW point + additions take place (instead of ~128 on average for a bit-by-bit approach). """ + WINDOW = 4 + def __init__(self, p): - self.table = [p] # table[i] = (2^i) * p - for _ in range(255): - p = p + p - self.table.append(p) + self.table = [] # table[i][j] = j * (2^(WINDOW*i)) * p + for _ in range((256 + self.WINDOW - 1) // self.WINDOW): + row = [GE()] + for _ in range((1 << self.WINDOW) - 1): + row.append(row[-1] + p) + self.table.append(row) + for _ in range(self.WINDOW): + p = p + p def mul(self, a): result = GE() a = a % GE.ORDER - for bit in range(a.bit_length()): - if a & (1 << bit): - result += self.table[bit] + mask = (1 << self.WINDOW) - 1 + for row in self.table: + result += row[a & mask] + a >>= self.WINDOW return result # Precomputed table with multiples of G for fast multiplication FAST_G = FastGEMul(G) + +@functools.lru_cache(maxsize=1024) +def mul_G(a): + return FAST_G.mul(a) From ca43addc41d814918fc44a70dbaf32a795573e1a Mon Sep 17 00:00:00 2001 From: Konstantin Akimov Date: Tue, 27 Oct 2026 18:04:36 +0700 Subject: [PATCH 4/4] fix(net): don't count disconnecting peers in CConnman::AddConnection Since 1df71a0f22 GetNodeCount and GetNodeStats skip peers with fDisconnect set, so getpeerinfo and getconnectioncount stop reporting a peer as soon as the node decides to drop it. The peer stays in m_nodes until the next DisconnectNodes pass of the socket handler thread, which can be up to 500 ms later. AddConnection still counted those lingering entries toward the per-type outbound limit. A caller that sees a peer disappear from getpeerinfo and then calls addconnection for the same connection type can therefore be rejected with "Already at capacity for specified connection type" even though no live connection of that type exists. The functional test framework does exactly this in disconnect_p2ps, and p2p_blocksonly.py hit it in CI: after two block-relay-only peers were disconnected for protocol violations, opening a third one failed while the two dead ones were still in m_nodes, and the test timed out waiting for a connection that was never made. Make AddConnection ignore peers marked for disconnect, matching the definition of a live connection already used by GetNodeCount and GetNodeStats. The outbound semaphore still bounds the total number of outbound connections, so a dying peer briefly overlapping with its replacement cannot exceed the overall limit. Bitcoin Core does not need this: its GetNodeStats lists peers until they leave m_nodes, so the framework's wait in disconnect_p2ps returns only after the slot is really free. --- src/net.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/net.cpp b/src/net.cpp index ef787baae989..e02d694c8156 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -2052,7 +2052,7 @@ bool CConnman::AddConnection(const std::string& address, ConnectionType conn_typ // Count existing connections int existing_connections = WITH_READ_LOCK(m_nodes_mutex, - return std::count_if(m_nodes.begin(), m_nodes.end(), [conn_type](CNode* node) { return node->m_conn_type == conn_type; });); + return std::count_if(m_nodes.begin(), m_nodes.end(), [conn_type](CNode* node) { return node->m_conn_type == conn_type && !node->fDisconnect; });); // Max connections of specified type already exist if (max_connections != std::nullopt && existing_connections >= max_connections) return false;