Repository navigation
feat(abci)!: add the ApplyBlockRandomness method and commit beacon fields - #1582
QuantumExplorer wants to merge 22 commits into
Conversation
…elds
Add a new ABCI method, ApplyBlockRandomness, for same-block randomness.
Tenderdash will call it at consensus version 2 with the random beacon of a
block and round (the BLS threshold signature recovered from its prevotes),
and the application returns the app hash the precommits sign.
- proto: RequestApplyBlockRandomness {hash, height, round, beacon} and
ResponseApplyBlockRandomness {app_hash} in the Request/Response oneofs and
the ABCI service; CommitInfo.beacon and CommitInfo.app_hash;
Commit.beacon and Commit.app_hash.
- Go: Application interface, socket, gRPC, routed and local clients, socket
server, proxy client with timing metrics, regenerated mocks.
- BaseApplication returns ErrApplyBlockRandomnessNotImplemented: an empty
app hash must never pass as an answer.
- kvstore example: returns the app hash of the processed block unchanged.
- types.Commit carries Beacon and AppHash through protobuf and CommitInfo.
MaxCommitOverheadBytes grows by 132 bytes (96+2 for the beacon, 32+2 for
the app hash).
Nothing calls the method yet, and the new fields stay empty, so behaviour is
unchanged apart from the larger commit overhead reserved by the mempool size
pre-check.
Refs dashpay/platform#4682
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
|
✅ Final review complete — no blockers (commit cd6b4ec) · triage: critical |
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
Commit.beacon and Commit.app_hash are covered neither by the commit hash nor by its threshold signature, so a peer relaying a validly signed commit could fill them with any bytes. Before this fix a block sync peer could pad the beacon of an honest commit: the node verified the signature, stored the padded commit as its seen commit and passed it to FinalizeBlock. Over gRPC ABCI a beacon above 4 MB made FinalizeBlock fail and the node crash on every restart, and a beacon larger than Block.MaxBytes left it unable to propose. Commit.ValidateBasic now refuses a commit unless both fields are empty or exactly 96 and 32 bytes, whatever the consensus version. Every path that accepts a commit on its own decodes it through CommitFromProto or validates it with ValidateBasic (block sync responses, consensus Commit messages, light blocks, stored commits), so such a commit is refused before it is stored or passed on. A block's LastCommit is not refused for these fields: releases before them skip them as unknown, so refusing such a block would split an upgraded node from them. BlockFromProto decodes it without the malformed fields instead, so they are never read, kept or passed on. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elds Releases before Commit.beacon and Commit.app_hash (fields 7 and 8) skip those fields whatever their wire type. The generated decoder of this release instead refuses the whole block when its LastCommit carries one with a wire type other than bytes. In a network still running older releases, a byzantine proposer could therefore get such a block committed by the older validators while upgraded nodes could not decode it, and an older node that stored it would panic loading it after upgrading. types.UnmarshalBlockProto decodes block bytes as before and, only when that fails, drops the mistyped occurrences of those two fields from the block's LastCommit and decodes again, so an upgraded node reads every block an older one does. Consensus block part assembly and BlockStore.LoadBlock use it. google.golang.org/protobuf becomes a direct dependency for its wire format parser. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example application looked up the state for ApplyBlockRandomness by height and round only. A node that executed two blocks of one round, as with an equivocating proposer whose second block the network commits after a later round was processed, failed with "more than one proposal executed", which the spec makes fatal, and a block that was not processed silently got another block's app hash. ProcessProposal now records which state each block hash left at each round, and ApplyBlockRandomness answers for the block the request names, refusing one that was not processed. Without a hash it still uses the only state of that height and round. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
…ck commits The ApplyBlockRandomness contract left four things open: - Vote extensions: Tenderdash calls ApplyBlockRandomness before ExtendVote, but other processes verify a Precommit's extensions against the state ProcessProposal left, possibly before they recover the beacon. Extensions and their verification must not depend on the beacon. - What the randomness may change: only the state behind the returned app hash. Transaction results, events and validator set or consensus parameter updates come from ProcessProposal and are fixed by the header the Prevotes signed. - FinalizeBlock: from consensus version 2 the committed state is the candidate with the commit's beacon applied, and its root must equal commit.app_hash, so an application in same-block execution mode reads the randomness fields as well as the hash. - When it is called: on every path that finalizes a block from a commit, the node's own included; the proto said only commits the node did not assemble itself. The ABCI call grammar gains the method, and the CommitInfo table links VoteExtension to its proto instead of a missing anchor. Regenerated types.pb.go and api.md (comments only). Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ABCI interface gained the ApplyBlockRandomness method and the CommitInfo beacon and app_hash fields. RELEASES.md makes interface changes a minor bump, and v1.8.0 to v1.8.2 already report 1.4.0, so an application built against these protos could not tell from RequestInfo.abci_version that a Tenderdash without them is too old. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… reserve MaxCommitOverheadBytes had grown by the 132 bytes of a commit's beacon and app hash, but its only runtime use is the mempool's per-transaction size pre-check, and commits carry those fields only from consensus version 2. Before that, every chain lost 132 bytes of transaction size at CheckTx, and one with Block.MaxBytes between 1129 and 1260 refused every transaction while its proposals still had room. MaxCommitOverheadBytes is back to the commit without the fields, and the new MaxCommitRandomnessBytes holds them, for the consensus version 2 check to add. Commit.String, StringIndented and its log fields now show the beacon and the app hash, so two commits that differ only there can be told apart. TestCommitBeaconFields encodes and decodes Commit and CommitInfo on the wire and compares whole structs. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestRouting routes ApplyBlockRandomness to its configured client and checks the response type, and TestApplyBlockRandomnessMetrics checks that the proxy client records the call under apply_block_randomness, for success and failure. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
Pin TENDERDASH_COMMITISH to c0e8c7f4, which carries the review fixes of dashpay/tenderdash#1582: ABCISemVer 1.5.0 and comments on ApplyBlockRandomness, FinalizeBlock and Commit.beacon. The messages are unchanged. The crate now reports ABCI 1.5.0, so its version check requires a Tenderdash with ABCI ^1.5. Cover that in the version matrix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
thepastaclaw
left a comment
There was a problem hiding this comment.
Final validation — Phase 1 + Phase 2
The compatibility decoder still rejects crafted block encodings accepted by the base revision, creating a rolling-upgrade consensus acceptance mismatch; proposal-side CommitInfo conversion also omits the new fields. Static verification at c0e8c7f confirms one blocking issue and one non-blocking suggestion; no builds or tests were run. Current CI reports successful build, tests, lint, vet, vulnerability and generation checks, with PR Hygiene still pending.
🔴 1 blocking | 🟡 1 suggestion(s)
1 finding(s) not shown inline (the lines are not part of this PR's diff)
🟡 Suggestion: Preserve the new fields in proposal-side CommitInfo conversion
internal/state/execution.go:778-783
Commit.ToCommitInfo now copies Beacon and AppHash, but buildLastCommitInfo independently constructs the old four-field representation. Both RequestPrepareProposal.LocalLastCommit and RequestProcessProposal.ProposedLastCommit use this helper, so they omit the new fields even when block.LastCommit contains them. This leaves a sibling ABCI conversion incomplete and duplicates a conversion that already has an owner, contrary to STYLE_GUIDE.md's Don't Repeat Yourself rule. It is non-blocking while the feature remains inactive. Keep the existing initial-height guard, delegate to ToCommitInfo, and add regression coverage for both proposal requests with populated commit fields.
return block.LastCommit.ToCommitInfo()
source: muse-spark-1.3-contributor (phase1-reviewer: general); gpt-6.1-sol (phase2-reviewer: general)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — This cross-cutting change adds custom peer-facing block deserialization in types/block_decode.go (UnmarshalBlockProto), wires it into proposal processing, and changes commit validation and malformed-field handling in types/block.go, directly affecting block acceptance and upgrade compatibility. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `types/block_decode.go`:
- [BLOCKING] types/block_decode.go:83-85: Match legacy skip semantics in the compatibility decoder
The fallback still accepts fewer encodings than the base revision's generated decoder. Append `22 02 3b 44` to an otherwise valid serialized block: this adds another LastCommit occurrence containing a field-7 start-group and field-8 end-group. The legacy Block decoder merges the occurrence, and its Commit decoder skips the unknown group because skipTypes tracks depth without matching group numbers. At this head, strict decoding rejects field 7's wire type, then protowire.ConsumeFieldValue rejects the mismatched end-group, so UnmarshalBlockProto fails. A field-7 varint containing nine 0x80 bytes followed by 0x02 exposes the same problem: the legacy skipper accepts its termination, while protowire rejects uint64 overflow. A Byzantine proposer can therefore propose bytes that legacy validators accept but upgraded validators cannot decode; upgrading a node that already stored such a block also makes BlockStore.LoadBlock panic. Make both fallback scanners preserve the legacy generated decoder's acceptance rules, with bounds checks, and add differential regression cases for these encodings.
In `internal/state/execution.go`:
- [SUGGESTION] internal/state/execution.go:778-783: Preserve the new fields in proposal-side CommitInfo conversion
Commit.ToCommitInfo now copies Beacon and AppHash, but buildLastCommitInfo independently constructs the old four-field representation. Both RequestPrepareProposal.LocalLastCommit and RequestProcessProposal.ProposedLastCommit use this helper, so they omit the new fields even when block.LastCommit contains them. This leaves a sibling ABCI conversion incomplete and duplicates a conversion that already has an owner, contrary to STYLE_GUIDE.md's Don't Repeat Yourself rule. It is non-blocking while the feature remains inactive. Keep the existing initial-height guard, delegate to ToCommitInfo, and add regression coverage for both proposal requests with populated commit fields.
Out-of-scope follow-up suggestions (1)
These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.
- Preserve authenticated original block bytes through block sync — The base revision already loses unknown fields and noncanonical encodings when prepareBlockResponse serves LoadBlock(...).ToProto(). The receiving blockApplier.Apply rebuilds the part set from that representation, and ValidatorSet.commitSignData rejects the resulting BlockID when its PartSetHeader differs from the authentic commit. A Byzantine proposer can therefore get a noncanonical block committed that honest peers cannot subsequently serve successfully through block sync. This is a concrete pre-existing availability issue, independent of the new randomness fields.
- Follow-up: Track a separate change to retain and verify original authenticated block bytes or parts through block sync, with a regression covering a committed block containing ignored protobuf wire data.
|
Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them. |
…hen dropping mistyped commit fields The fallback in UnmarshalBlockProto drops a LastCommit's beacon and app hash fields that do not have the bytes wire type, so an upgraded node decodes every block an older one does. It split the bytes with protowire, which refuses encodings the generated decoder of older releases skips: a group closed by an end-group of another field number, and a ten-byte varint whose last byte overflows 64 bits. A proposer could then send a block that older validators accept and upgraded ones cannot decode. The scanner now reads tags, varints and skipped fields with the rules of the code gogoproto generates. The test checks each case against gogotypes.Empty, whose generated decoder skips every field as older releases skip fields 7 and 8, both for blocks it accepts and for blocks it still refuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quests buildLastCommitInfo built RequestPrepareProposal.LocalLastCommit and RequestProcessProposal.ProposedLastCommit field by field and left out the commit's beacon and app hash. It now keeps its initial-height guard and delegates to Commit.ToCommitInfo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the source-derived consensus specification (#1562). Conflict in spec/abci++/abci++_tmint_expected_behavior.md: v1.8-dev drops the Commit call from the ABCI grammar, and this branch adds check-commit and apply-randomness. The resolution keeps both, and notes that from consensus version 2 the state FinalizeBlock persists has the root ApplyBlockRandomness returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
TestMaxProposalBlockSize gives CreateProposalBlock a commit with a beacon and an app hash, which the proposal now drops, so its LastCommit and the block are back to their sizes without them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
Independent static verification of the complete diff at 2c9287d found no actionable in-scope defects, and all three prior findings are addressed. Current CI reports successful build, unit, integration, lint, protobuf, mock, and vulnerability checks; only PR Hygiene remains pending, and no local builds or tests were run. The base-only CBOR dependency update does not supersede this work, and the supplied merge evidence reports no conflicts.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The large, cross-cutting diff adds intricate peer-facing block deserialization in types/block_decode.go (UnmarshalBlockProto and custom protobuf wire parsing) and changes commit validation in types/block.go, directly affecting block acceptance and compatibility. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
…rsion 2 Neither field is covered by the commit hash or its threshold signature, and both are empty before consensus version 2, which this release does not define. A peer relaying a validly signed commit could still add correctly sized ones, which nodes stored as the seen commit, passed to FinalizeBlock and served on; a proposer could put them in its block's LastCommit, which reached ProcessProposal. Drop both where a commit is adopted (consensus Commit messages, block sync, state sync and its backfill) and from the last commit passed in proposal requests. One test now covers the proposer side, with a block that keeps the fields as a negative control for older releases' block sync, replacing the two overlapping ones; another covers block sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fallback allocated a copy of the whole block and one buffer per LastCommit occurrence before it knew whether anything had to be dropped. It now allocates nothing until the first field it drops, then writes everything into one buffer: 1 allocation instead of about 1M on 2 MiB of mistyped occurrences, and none when nothing is dropped. Varints are read with gogoproto's proto.DecodeVarint, which matches the generated code, instead of a copy of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tenderdash always sends the block hash, so the height and round fallback for requests without one served only tests. ProcessProposal now records every block under its round and hash, and the lookup has one path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- spec/core/data_structures.md, consensus.md, consensus-commit.md: the Commit fields Beacon and AppHash, their size rule, and how a block's LastCommit with malformed ones is read. - Requirement 10 and the ABCI overviews: ApplyBlockRandomness must not modify the committed state, since it can run for rounds that never commit. - Commit.app_hash and CommitInfo.app_hash: the commit signs it only through its state ID. - ApplyBlockRandomness: this release defines consensus versions 0 and 1 only, so it never calls it. - UPGRADING.md: the new method, BaseApplication's error, ABCI 1.5.0, the commit size rule and routing with the routed transport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
Resolve the conflict in abci++_basic_concepts.md with the mdBook docs migration: keep the ApplyBlockRandomness bullet next to the renamed ExtendVote one, and say Tenderdash in the lines this branch added, as v1.8-dev now does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Bots are done — your move: post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
Static verification of the complete diff found no actionable in-scope defects; all three prior findings are addressed. Current CI shows the build, all six unit-test shards, ABCI tests, lint, generated-artifact checks, and both e2e variants passing; only PR Hygiene remains pending. No local builds or tests were run, and the working tree is unchanged.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The cross-cutting change directly alters peer-facing block deserialization in types/block_decode.go (UnmarshalBlockProto and its custom protobuf wire parser) and commit validation in types/block.go, making backward-compatible block acceptance an intricate critical surface. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
dev moved the ABCI specs from spec/abci++ to spec/abci and shortened the proto comments so the contracts live in methods.md. This carries the ApplyBlockRandomness text to the new files, keeps dev's short comments on ExtendVote and FinalizeBlock with one randomness sentence each, and regenerates types.pb.go and api.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
Static verification at 9e4b04d confirms one blocking omission: proposer-supplied randomness survives canonical commit persistence and is published through commit APIs before activation. The three prior findings are fixed, and the block-sync wire-preservation issue is pre-existing and belongs in a separate follow-up. No local builds or tests were run; current CI reports successful code checks, with PR Hygiene pending.
🔴 1 blocking
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The cross-cutting change adds intricate peer-facing block deserialization in types/block_decode.go (UnmarshalBlockProto and its wire-parsing helpers), altering acceptance of commit encodings across protocol versions. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `types/block.go`:
- [BLOCKING] types/block.go:1079-1080: Normalize randomness before persisting the canonical last commit
The new serialization exposes a persistence path that bypasses pre-version-2 normalization. A Byzantine proposer can include an otherwise valid LastCommit with arbitrary 96-byte Beacon and 32-byte AppHash values: BlockFromProto retains correctly sized fields, and LastCommit validation does not authenticate either value. internal/store/store.go:605-607 then serializes block.LastCommit directly into blockCommitKey(height-1). LoadBlockCommit returns those injected fields, /commit publishes them as canonical commit metadata, and statesync.fetchLightBlock serves them to peers. Stripping received standalone commits and proposal-construction inputs does not sanitize a Byzantine proposer's LastCommit. Keep the generic ToProto conversion, but apply WithoutRandomBeacon when persisting the separate canonical commit record, without modifying the block or its committed parts. Add a regression that saves a block containing correctly sized injected fields and verifies that LoadBlockCommit returns empty randomness fields while the original block parts remain unchanged.
Out-of-scope follow-up suggestions (1)
These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.
- Preserve committed block bytes throughout block sync — Block-sync serving decodes stored parts with LoadBlock and reserializes through ToProto; the receiving applier reconstructs parts with MakePartSet. Unknown protobuf fields or noncanonical field ordering accepted during consensus are lost or normalized, producing a different PartSetHeader from the committed one and preventing block sync past that height. Both behaviors already exist at the exact review base, independently of this PR.
- Follow-up: Address this in a separate follow-up PR by preserving the committed wire representation through block-sync serving and verification, or introducing an upgrade-gated canonical-encoding rule. Include regressions for unknown fields and reordered protobuf fields.
| c.Beacon = commit.Beacon | ||
| c.AppHash = commit.AppHash |
There was a problem hiding this comment.
✅ Resolved at
cd6b4ec8; see the replies below.
🔴 Blocking: Normalize randomness before persisting the canonical last commit
The new serialization exposes a persistence path that bypasses pre-version-2 normalization. A Byzantine proposer can include an otherwise valid LastCommit with arbitrary 96-byte Beacon and 32-byte AppHash values: BlockFromProto retains correctly sized fields, and LastCommit validation does not authenticate either value. internal/store/store.go:605-607 then serializes block.LastCommit directly into blockCommitKey(height-1). LoadBlockCommit returns those injected fields, /commit publishes them as canonical commit metadata, and statesync.fetchLightBlock serves them to peers. Stripping received standalone commits and proposal-construction inputs does not sanitize a Byzantine proposer's LastCommit. Keep the generic ToProto conversion, but apply WithoutRandomBeacon when persisting the separate canonical commit record, without modifying the block or its committed parts. Add a regression that saves a block containing correctly sized injected fields and verifies that LoadBlockCommit returns empty randomness fields while the original block parts remain unchanged.
source: gpt-6.1-sol (phase2-reviewer: general)
There was a problem hiding this comment.
Resolved (re-reviewed at cd6b4ec8): Your saveBlockToBatch change strips both fields before writing the previous height's canonical commit, while preserving the original block parts. The new regression verifies canonical reload and confirms that saving does not mutate the supplied block.
|
Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them. |
Nothing signs the beacon and app hash of a block's LastCommit, so a proposer can fill them with well-sized values. SaveBlock stored them as the canonical commit of the previous height, and LoadBlockCommit then served them through /commit and state sync. Drop them from that record. The block's parts keep them, as the encoding the next commit signs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
Static verification of the complete diff at cd6b4ec found no actionable in-scope defects. Three prior findings are fixed; the proposal-side field-preservation request is withdrawn because forwarding unauthenticated randomness before consensus version 2 would violate the intended boundary. Current CI code checks passed, with PastaClaw review and PR Hygiene pending; no local builds or tests were run.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The cross-cutting diff adds intricate peer-facing block deserialization in types/block_decode.go (UnmarshalBlockProto and custom protobuf field parsing) and changes commit validation in types/block.go, directly affecting acceptance compatibility between node versions. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— tenderdash-consensus-security (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
|
Your move: thepastaclaw left review threads unresolved; resolve them. |
Basic explanation
Dash Platform wants fair same-block randomness for games: a card's rarity or a token payout that nobody can predict or pick, set in the block that asks for it. The random value comes from the validators: when enough of them prevote for a block, their BLS signature shares combine into one threshold signature that nobody could compute in advance. That signature is the random beacon.
This PR adds the ABCI method that hands the beacon to the application, and the commit fields that will carry it. Nothing calls the method yet; a follow-up PR switches it on at consensus version 2.
Before, the application had no way to learn the beacon, and a commit carried only its threshold signature:
After, there is a new call, and commits have room for the beacon and the app hash the precommits sign:
TL;DR: Add the
ApplyBlockRandomnessABCI method andbeacon/app_hashfields on commits. Nothing calls the method yet, and nodes keep both fields empty before consensus version 2.User story
As a Dash Platform developer, I want Tenderdash to give Drive a random beacon for each block, so that game contracts can get unpredictable values in the same block that requests them.
Scenario
Base flow
A Platform contract declares a random property. A user creates a document; the block runs it with a placeholder. Once the prevotes for the block reach the threshold, the beacon exists, and Drive should replace the placeholder with a value derived from it before the precommits sign the final state.
Actual behavior
There is no ABCI call that carries a beacon to the application, and the commit has nowhere to record the beacon or the final app hash.
Expected behavior
The ABCI interface, clients and commit format can carry the beacon and the final app hash. The consensus change that uses them comes in the stacked follow-up PR.
Detailed discussion
What was done
RequestApplyBlockRandomness {hash, height, round, beacon}andResponseApplyBlockRandomness {app_hash}, added to theRequest/Responseoneofs and theABCIApplicationservice;CommitInfo.beacon = 5andCommitInfo.app_hash = 6;Commit.beacon = 7andCommit.app_hash = 8. Regenerated Go code, mocks andspec/abci++/api.md; documented the method inspec/abci++/abci++_methods.md(and brought its staleCommitInfotable in line with the proto).Application.ApplyBlockRandomness; socket, gRPC, routed and local clients; socket server; proxy client with a timing metric (apply_block_randomness).BaseApplication.ApplyBlockRandomnessreturnsErrApplyBlockRandomnessNotImplemented. Every other base method returns an empty response, but an emptyapp_hashmust never pass as an answer.types.Commit: newBeaconandAppHashfields, carried through protobuf andToCommitInfo(JSONomitempty, so RPC output is unchanged while they are empty). The newMaxCommitRandomnessBytes(132: 96+2 for the beacon and 32+2 for the app hash, including their protobuf tags) is what a commit grows by when it carries them;MaxCommitOverheadBytesis unchanged, so the mempool reserves nothing more until the follow-up adds it at consensus version 2.abci++_methods.md, the call grammar inabci++_tmint_expected_behavior.md) say that vote extensions and their verification must not depend on the beacon, that the randomness may change only the state behind the returned app hash (transaction results, events and validator set or consensus parameter updates stay those ofProcessProposal), that from consensus version 2FinalizeBlockcommits the candidate with the commit's beacon applied and its root must equalcommit.app_hash, and that the method is called on every finalizing path, a validator's own commit included.ABCISemVer1.4.0 → 1.5.0, asRELEASES.mdasks for an interface change.Commit.String,StringIndentedand the commit's log fields show the beacon and app hash.beaconorapp_hash, so a peer relaying a validly signed commit could fill them with any bytes.Commit.ValidateBasicnow refuses a commit unless both are empty or exactly 96 and 32 bytes (ErrInvalidCommitRandomness), whatever the consensus version. Every path that accepts a commit on its own goes through it (block sync responses, consensusCommitmessages, light blocks, stored commits), so a padded commit is refused before it is stored or passed on. A block'sLastCommitis not refused for them, because releases before this one skip the fields as unknown and refusing the block would split upgraded nodes from them:BlockFromProtoreads it without the malformed fields instead.types.UnmarshalBlockProtodecodes as before and, only when that fails, drops those mistyped occurrences from the block'sLastCommitand decodes again; it allocates nothing unless it drops a field, and then one buffer. Consensus part assembly andBlockStore.LoadBlockuse it, so during a rolling upgrade an upgraded node decodes every block an older one commits.google.golang.org/protobufbecomes a direct dependency for its wire format writer.LastCommit. Nodes drop both (Commit.WithoutRandomBeacon) where they adopt a commit (consensusCommitmessages, block sync, state sync and its backfill), from the commit a proposal builds on (CreateProposalBlock, so older releases, which drop the fields when they decode a block and rebuild its parts, can still block sync it), and from the last commitPrepareProposalandProcessProposalpass to the application. The follow-up keeps and verifies them from consensus version 2.Committables inspec/core/data_structures.md,spec/consensus/consensus.mdandconsensus-commit.mdlist the new fields and their rules; Requirement 10 inabci++_app_requirements.md,abci++_basic_concepts.mdand the adapting-applications list coverApplyBlockRandomness, which must not modify the committed state; the proto and spec say the commit signsapp_hashonly through its state ID;UPGRADING.mdhas an entry for the change, including routing the method with the routed ABCI transport.Nothing calls the new method and honest nodes never set the new fields, so consensus behaviour is unchanged. The visible effects are that a commit with a wrongly sized beacon or app hash is refused, that a correctly sized one a peer or proposer adds is dropped instead of being stored or passed to the application, and that
RequestInfo.abci_versionis 1.5.0.Before the size bound, a block sync peer could serve an honest commit with a 5 MiB beacon:
After, the response is refused before anything is stored:
Testing
Targeted tests on the touched packages:
TestClientServernow callsApplyBlockRandomnessover both the socket and the gRPC transports. Regression tests for the review fixes:TestBlockResponseRefusesPaddedCommitBeacon: a block sync response whose signed commit has its beacon padded (25 MiB, 5 MiB) or an app hash without a beacon is refused withErrInvalidCommitRandomness.TestCommitMessageRefusesMalformedRandomnessandTestCommitValidateBasicRandomnessSizes: the same for consensusCommitmessages and every size combination.TestBlockFromProtoMalformedLastCommitRandomness: a block whoseLastCommithas malformed fields still decodes, without them, and keeps its hash.TestUnmarshalBlockProtoMistypedLastCommitRandomnessandTestLoadBlockMistypedLastCommitRandomness: blocks whoseLastCommitcarries field 7 or 8 as a varint, fixed32 or fixed64 decode (and load from the store) as an older release reads them; truncated bytes still fail.TestCommitBeaconFieldsencodes and decodesCommitandCommitInfoon the wire and compares the whole structs;TestRoutingandTestApplyBlockRandomnessMetricscover the routed client and the proxy client'sapply_block_randomnessmetric.TestApplyBlockRandomnessEquivocation: the kvstore answers for the requested block after an equivocating proposer's two blocks in one round, and refuses a block it did not process.TestProposalDropsRelayedCommitRandomness: a proposal built from a relayed commit with a correctly sized beacon and app hash drops them and keeps the parts an older release rebuilds (a block that kept them would not), and neitherPrepareProposalnorProcessProposalpasses them to the application.TestBlockSyncDropsRelayedCommitRandomness: block sync of a commit with a correctly sized beacon and app hash added neither passes them toFinalizeBlocknor stores them in the seen commit.make proto-gen,make mockeryandmake lint(new findings againstv1.8-dev) are clean.Breaking changes
Applicationinterface has a new method. Go applications that embedBaseApplicationcompile unchanged, but get an error if Tenderdash ever calls the method, which it will only do from consensus version 2. Applications must implement it before their chain moves to consensus version 2.ABCISemVeris 1.5.0. SeeUPGRADING.md.beaconorapp_hashis set with any size other than 96 and 32 bytes is refused, at every consensus version. No release sets them before consensus version 2.Prior work
Attribution
Prepared with Claude Code.
🤖 Generated with Claude Code
PR Hygiene ·
cd6b4ec/self-reviewedUPGRADING.md,abci/client/grpc_client.go,abci/client/mocks/client.goand 41 more) — lklimek or shumkovWhen every merge requirement is met, the
PR Hygienecheck passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.