Skip to content

feat(validator_store): detach non-builder envelope signing from the Lighthouse callback - #1

Merged
shane-moore merged 2 commits into
envelope-no-qbftfrom
feat/envelope-non-builder-trigger
Sep 3, 2026
Merged

feat(validator_store): detach non-builder envelope signing from the Lighthouse callback#1
shane-moore merged 2 commits into
envelope-no-qbftfrom
feat/envelope-non-builder-trigger

Conversation

@shane-moore

@shane-moore shane-moore commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Stacked on sigp#1286 (envelope-no-qbft); opened on the fork with that base so the diff is only the delta. Re-target to sigp/anchor:epbs after sigp#1286 merges. Tracks sigp#1287.

Problem, Evidence, and Context (Required)

  • A non-builder's envelope share is signed only when Lighthouse calls sign_execution_payload_envelope, and Lighthouse reaches that call only after fetching the operator's own envelope from its beacon node (block_service.rs:668-680 at pin e58ec88fe, ?-propagated). The node serves that GET from a cache written only for locally built payloads (block_production/gloas.rs:709-741; a gossip-cached external bid wins with payload_data: None, 1010-1020). So an operator whose node took an external bid while the cluster decided a self-build block never signs its share. Stop external-builder envelope 404 after block publication sigp/lighthouse#9948 turns that 404 into a skip; same loss.
  • The non-builder path needs nothing from that callback: it signs the disseminated envelope against its own record of the decision. Only the trigger was coupled.
  • Reachable at the pin (gossip bid selection at gloas.rs:916-935; --builder-boost-factor is node-wide config), not observed on any devnet (ssv-mini has no builders). Planning note: Envelope duty trigger keyed on the local block in Lighthouse block service sigp/anchor#1287.

Change Overview (Required)

  • sign_block spawns sign_disseminated_envelope after the decided block's threshold signature succeeds, when the recorded context has a self-build bid on a block another operator built. The task awaits the dissemination to the payload-due deadline, validates it, and contributes the partial signature. Detached, bounded by the deadline, terminal failure logged at warn.
  • The Lighthouse callback on a non-builder returns a new EnvelopeNonBuilderDelegated sentinel immediately instead of waiting; EnvelopeNotBuiltLocally had no other producer and is deleted.
  • Weak<Self> via Arc::new_cyclic gives the &self trait method something to spawn with.
  • Unchanged: the builder path, dissemination validation and store, the wire, outcome metric labels.

Risks, Trade-offs, and Mitigations (Required)

  • Spawn happens after block signing, not at decide time: an operator whose own block signing fails while the cluster reaches threshold contributes no envelope share. Unobserved. This placement is load-bearing, see the repeat-sign_block bullet below.
  • Covers non-builders only. The builder still depends on Lighthouse's fetch and publish (proposer-nodes routing is Lighthouse's open TODO in the same function). A non-builder whose Lighthouse never reached sign_block still has no context and no task.
  • A repeat sign_block for the same slot is real, not hypothetical: the devnet run below logged one Unable to sign block: SameData at slot 101. The guard is the spawn placement. SameData comes from the slashing-protection check inside sign_abstract_block, so the second call returns at the ? before reaching the spawn and creates no second task. Counters confirmed no operator exceeded one outcome per duty. Spawning at decide time instead would double-sign at exactly that slot.
  • Lighthouse still logs one recoverable error per non-builder proposal; now immediate rather than after the wait.

Validation (Required)

  • anchor_validator_store: 166 tests green (was 162). New: mixed-bid regression (sign_block on a local external-bid block with a fixed self-build decision spawns the task, which signs the disseminated root; a later callback returns the sentinel and adds no second share), builder and external-decision negatives with a valid dissemination pre-stored so a wrongly spawned task would be caught, callback-returns-without-waiting under a paused clock. Non-builder worker tests drive the task body directly.
  • make cargo-fmt-check, make lint clean.
  • ssv-mini devnet, all-Anchor Gloas profile, 4 operators split across 2 beacon nodes so every proposal has both builders and non-builders. Baseline is feat: replace envelope QBFT with disseminate-and-sign (SIP-94 §6) sigp/anchor#1286's own run on the same host and params, differing only in the Anchor image tag. 23 managed proposals, 23 of 23 envelopes revealed on chain, zero task failures, zero duplicate signatures. signed_other + published == received exactly on every operator, and delegated == lh_sentinel == signed_other: one outcome per node per duty.
  • Two results worth calling out:
    • The share provably comes from the task. At slots 101 and 123 an operator logged Signed another operator's envelope before its own Received execution payload envelope. The callback path could not produce that ordering, since it only runs after the fetch returns.
    • The repeat-sign_block case fired and was handled. See the risk bullet above.
  • Faults: operator stopped for 65 slots, 8 duties ran at the exact 3-of-4 threshold and all 8 revealed; on restart the operator resumed in both roles. Resources flat, OS thread count 67 to 68 throughout, no OOM, no unexpected restarts.
  • Still not reproducible on this network: the mixed-bid trigger itself (needs a real ePBS builder on one beacon node), the task-side deadline miss, and the task-side dissemination timeout. All three are covered by unit tests.
  • Evidence: pr-reviews-style workbench at ssv-mini/evidence-envelope-non-builder-20260902/ (README with the 11-row oracle matrix and verdicts, per-duty chains, regression ticks, 89 resource samples, fault log).

Rollback (Required for behavior or runtime changes; optional otherwise)

Blockers / Dependencies (Optional)

Additional Info / Next Steps (Optional)

  • Plan and Codex review ledger: plans/anchor/2026-09-02-issue-1287-non-builder-envelope-trigger.md (vault). The planning recommendation was to hold this until a builder-enabled network exists. The devnet run above supersedes that: the mixed-bid trigger still cannot be reproduced, but the mechanism it relies on is now validated over 23 duties plus two faults, and the repeat-sign_block finding turned the plan's one open concurrency question from speculative into observed and handled.

Sign a non-builder's SIP-94 §6 envelope share from a task spawned in
sign_block once the decided block is threshold-signed, instead of from
Lighthouse's envelope callback. That callback first fetches the
operator's own envelope from its beacon node and never reaches the
store when the node holds none (its local bid was external while the
cluster decided a self-build block), so the share was lost. The
callback now returns a delegated sentinel for non-builder contexts;
the builder path is unchanged. Tracks sigp#1287.
@shane-moore
shane-moore marked this pull request as ready for review September 2, 2026 23:28
Lighthouse can call sign_block twice for one slot: a second block-service
notification for the same slot hit three of four operators at one devnet
slot, about 12 s after the first call. The non-builder envelope task is
spawned only after sign_abstract_block, so the repeat is rejected by
slashing protection as SameData before it can spawn a second task.
Nothing pinned that placement: the existing spawn-path tests disable
slashing protection and call sign_block once. Add a slashing-enabled test
that calls sign_block twice and expects one envelope collection; it fails
with two collections if the spawn is moved above the slashing check.

Also blind the local envelope only on the builder path. The non-builder
callback now returns before using it, so hashing the full payload up
front was wasted work on every non-builder duty.

@shane-moore shane-moore left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at a64a884 against cb50756. No remaining code-review objections. The repeated sign_block test detects the load-bearing spawn placement, and local_blinded is now restricted to the builder path. The QueueFull disposition is appropriate for this PR and belongs in shared collector/processor handling. Local validator_store tests and formatting pass. Ready to merge after CI passes on this head.

Reviewed by GPT-5.6 Sol (xhigh).

@shane-moore
shane-moore merged commit 61f0641 into envelope-no-qbft Sep 3, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant