mempool: add new OrphanManager, and PolicyEnforcer based on new txgraph #2438
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add GetConflicts method to detect which mempool transactions would be replaced by a new transaction. Returns both individual conflicting transactions and their packages to support package-based eviction. Uses the spentBy index for O(1) conflict lookups per input.
Add tests covering all conflict scenarios including single conflicts, multiple conflicts, descendant cascading, and partial conflicts.
Add benchmarks for various conflict detection scenarios to validate O(1) lookup performance.
This commit introduces the OrphanManager, which manages orphan transactions using a dedicated TxGraph instance. An orphan is a transaction whose inputs reference outputs that are not yet confirmed or in the mempool. The key architectural decision is using a separate graph instance rather than storing orphans in the main mempool graph. This separation provides several critical benefits. First, it maintains isolation between validated mempool transactions and potentially invalid or spam orphans. Second, it enables different lifecycle management, as orphans have TTL-based expiration and peer-based tagging that don't apply to confirmed mempool transactions. Third, despite being separate, the graph structure still enables efficient package tracking, allowing us to identify all descendant orphans when a parent transaction arrives. The OrphanManager provides comprehensive orphan lifecycle management. AddOrphan validates size and count limits before adding transactions tagged with their sending peer. RemoveOrphan supports both isolated removal and cascading removal of descendants, with proper metadata cleanup for all affected transactions. RemoveOrphansByTag enables efficient cleanup of all orphans from a misbehaving or disconnected peer using the byTag index. ExpireOrphans implements lazy TTL-based expiration with configurable scan intervals to prevent memory exhaustion from stale orphans. The ProcessOrphans method implements batch orphan promotion using BFS traversal. When a parent transaction arrives, it efficiently identifies all orphan descendants and attempts to promote them through a provided acceptance function. The implementation uses the Queue collection from txgraph for clean traversal and tracks visited nodes to handle complex graph structures like diamond patterns where a transaction has multiple parents. All operations use structured errors (ErrOrphanAlreadyExists, ErrOrphanTooLarge, ErrOrphanLimitReached, ErrOrphanNotFound) that can be checked with errors.Is for robust error handling. The implementation uses RWMutex for thread safety, allowing concurrent reads while serializing writes.
This commit adds a thorough test suite for the OrphanManager covering all core functionality and edge cases. The tests verify proper behavior across the complete orphan lifecycle from addition through expiration and promotion. The suite includes nine test functions covering distinct aspects of orphan management. TestOrphanManagerAddOrphan verifies basic addition with enforcement of size and count limits, including proper error handling for duplicates and limit violations using structured error assertions with errors.Is. TestOrphanManagerSizeLimit specifically validates rejection of oversized transactions to prevent memory exhaustion attacks. TestOrphanManagerRemoveOrphan verifies both cascade and non-cascade removal behavior, ensuring that descendant metadata is properly cleaned up when using cascade mode while preserving disconnected descendants when not cascading. TestOrphanManagerRemoveOrphansByTag confirms efficient peer-based removal using the tag index, critical for handling misbehaving or disconnected peers. TestOrphanManagerExpireOrphans validates the lazy TTL-based expiration mechanism, ensuring orphans are removed after their configured lifetime while respecting the scan interval to avoid excessive CPU usage. TestOrphanManagerGetOrphan verifies the query interface for retrieving specific orphans. The most complex tests cover orphan promotion scenarios. TestOrphanManagerProcessOrphans builds a multi-branch orphan tree and verifies that when a parent arrives, all descendant orphans are correctly identified and promoted through the acceptance function. TestOrphanManagerProcessOrphansPartialFailure ensures that when some orphans fail validation, successful orphans are still promoted while failures remain in the pool for potential future promotion. TestOrphanManagerPackageTracking validates that the graph structure correctly tracks complex orphan package relationships including diamond patterns where transactions have multiple parents. The tests use a helper function createTestOrphanTx that generates unique transactions using a counter to ensure deterministic but distinct transaction hashes across test runs.
In this commit, we introduce the StandardPolicyEnforcer, a new component that separates policy decisions from graph data structure operations. This clean separation enables easier testing, different policy configurations, and clearer code boundaries between what the mempool stores versus what policies it enforces. The PolicyEnforcer interface defines five core validation methods that work against a minimal PolicyGraph interface. This abstraction allows policy enforcement to operate on any graph-like structure without tight coupling to the specific txgraph.Graph implementation. The StandardPolicyEnforcer implements Bitcoin Core-compatible policies: RBF (Replace-By-Fee) Support: We implement full BIP 125 RBF validation including both explicit signaling (sequence numbers ≤ 0xfffffffd) and inherited signaling where transactions inherit replaceability from unconfirmed ancestors. The recursive ancestor traversal uses a cache to avoid redundant graph walks when checking deep transaction chains. The ValidateReplacement method enforces all five BIP 125 rules: eviction limits, no parent spending, higher fee rates, sufficient absolute fees, and no new unconfirmed inputs. This matches Bitcoin Core's logic and ensures compatibility with the existing network. Ancestor/Descendant Limits: Bitcoin Core limits transaction chains to 25 ancestors and 25 descendants, each with a maximum total size of 101 KB, to prevent unbounded chain growth in the mempool. We implement identical limits with clear error messages that specify which limit was exceeded and by how much. Fee Rate Validation: The ValidateRelayFee method implements minimum relay fee checking with an exponentially decaying rate limiter for low-fee transactions. This prevents spam while allowing some free transactions through, using the same 10-minute half-life decay as Bitcoin Core. The PolicyConfig structure provides sensible defaults matching Bitcoin Core but allows operators to customize limits for different network conditions or use cases. All policy violations return specific error types that enable callers to distinguish between different rejection reasons.
In this commit, we add extensive test coverage for the StandardPolicyEnforcer including unit tests for each validation method and property-based tests to verify correctness across a wide range of inputs. The test suite uses a mockGraph implementation that provides the minimal PolicyGraph interface needed for testing. This allows us to test policy logic in isolation without requiring a full txgraph.Graph implementation, making tests faster and easier to understand. Unit Test Coverage: We test each major feature independently with focused scenarios. For RBF signaling, we verify both explicit signaling via sequence numbers and inherited signaling through multiple generations of ancestors. The deep inheritance test confirms that signaling propagates correctly through arbitrarily long transaction chains. The replacement validation tests cover all BIP 125 rules including edge cases like insufficient fee rates, too many evictions, and attempts to spend from parent transactions. Each test uses realistic transaction structures with proper fee calculations. The ancestor and descendant limit tests verify both count and size limits, ensuring we correctly enforce Bitcoin Core's 25-transaction and 101 KB limits. We test both at-limit (passing) and over-limit (failing) scenarios. Fee validation tests cover the minimum relay fee logic and the exponentially decaying rate limiter for low-fee transactions. The rate limiter tests verify both rejection when limits are exceeded and proper decay over time. Property-Based Testing: Using the rapid library, we add three property-based tests that verify invariants across randomized inputs. TestPropertySignalsReplacementTransitive confirms that RBF signaling is correctly inherited through chains of random depth. TestPropertyAncestorLimitEnforced generates chains of varying lengths to ensure limits are enforced at exactly the configured threshold. TestPropertyFeeRateMonotonic verifies fee rate calculations are consistent. These property tests run 100 iterations each with randomized parameters, providing high confidence that the policy enforcement logic handles edge cases correctly. Test coverage: 80-100% across all PolicyEnforcer methods.
Pull Request Test Coverage Report for Build 18179085146Details
💛 - Coveralls |
kmk142789
approved these changes
Nov 5, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this commit, we a do some prep before updating the existing mempool to use the new tx graph, which has first class support for concepts like tracking packages, etc -- as the entire mempool is organized into an in-memory DAG.
First, we create a new
OrphanManagerthat implement the existing orphan logic, using the newtxgraphpackage. Importantly it supports being able to extract conflicts as packages, which will be useful for future changes.Second, we create a new abstract
PolicyEnforcerinterface. This is meant to decouple the policy logic, from the normal mempool logic. We're now able to fully test all the policy logic independent of the existing mempool struct.The next PR in the series will create a new parallel mempool implementation, along with tests. After that we'll create some equivalence tests to verify that no major regressions occur w.r.t functionality. This'll likely use tee'd transactions from the live network.
After that PR, we'll implement new policies for: truc, 1p1c, eph dust, etc. This'll build on the existing base we've created.