feat: migrate producer to upstream create-snapshots + bootstrap#42
Merged
Conversation
Repoint flake from lambdasistemi/amaru/main to lambdasistemi/amaru/feat/antithesis-testnet-support. The new branch carries the same 11-patch stack (runtime testnet params, era-history flags, ledger rotation/forecast/saturation fixes) rebased onto pragma-org/amaru@bc90e534 — 65 commits ahead of the prior base. Only flake.nix amaru ref and flake.lock amaru entry change. iohkNix and blst are unchanged — the upstream supranational/blst tarball shifted content under our pinned rev, but CI's fetch path still returns v0.3.14 (matching the lock); locally only override-input works around the regional content shift. Bumping iohkNix to match local would force CI off the v0.3.14 content. Local builds verified via override-input.
paolino
force-pushed
the
chore/bump-amaru-rebased
branch
from
0599cc2 to
bab6ba6
Compare
Contributor
Author
|
The
This is orthogonal to the amaru rebase. The image was built locally PR stays draft until either (a) the blst pin is moved to a stable |
Replace the removed ledger-state-emitter/convert-ledger-state/import-* pipeline with amaru create-snapshots (--targets-file/--cardano-db-dir to bypass Koios/Mithril) followed by amaru bootstrap, which derives nonces and imports packaged headers. Pin amaru to feat/testnet-bootstrap. - bootstrap-producer.sh: targets.json + snapshots.json from header-extractor, isolated cardano-db view (don't mutate the operator chain DB), era-history sidecars + bundle era-history.json, run bootstrap from the staging CWD. - new bundle contract: ledger.<net>.db + chain.<net>.db + era-history.json (nonces/headers baked into chain.db by bootstrap). - checks: db-analyser on producer PATH; reframe short-epoch samples/golden to producer + amaru run --era-history-file; grow short-epoch corpus to 3000 slots. - bats: rewrite sparse-boundaries/history for the new pipeline.
Two deployment-path bugs the create-snapshots/bootstrap producer migration introduced for the relay image (not covered by the nix checks, caught by the live short-epoch follow cluster): - bootstrap-producer-image: add db-analyser to runtimeInputs + contents. create-snapshots shells out to db-analyser, which was absent from the image, so the relay's bootstrap looped forever with 'db-analyser was not found'. - amaru-relay-bootstrap: bundle_complete no longer requires nonces.json; nonces + bootstrap headers are baked into chain.<net>.db by 'amaru bootstrap'. The stale check made the relay re-bootstrap endlessly and never exec amaru. Verified: amaru-relay-1 follows a live epoch-120 short-epoch chain past slot 1088 (epochs 3..9) with zero fatal errors.
paolino
changed the title
test-bootstrap-producer-live.bats still asserted nonces.json / snapshots/*.cbor / headers/* — the pre-bootstrap contract. amaru bootstrap bakes nonces+headers into chain.<net>.db; assert era-history.json + >=3 ledger snapshot dirs instead. This is the only check gated behind Docker (Live Bootstrap Producer CI), so it was missed by the build-gate.
The lock pinned blst at the v0.3.15 rev (6d960cd) but recorded v0.3.14's narHash; GitHub now serves v0.3.15 content for that rev in some regions, intermittently breaking the lock both locally (always) and in CI (flaky: Smoke Test failed on narHash mismatch). Pin iohk-nix's blst input to the v0.3.14 tag rev (8c7db7fe), which iohk-nix's crypto overlay expects and which hashes deterministically. Removes the need for the local --override-input.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps amaru to upstream pragma-org/amaru main and migrates the bootstrap producer off the removed ledger-state-emitter/convert-ledger-state/import-* commands onto upstream's create-snapshots + bootstrap.
amaru pinned to lambdasistemi/amaru@feat/testnet-bootstrap (upstream main + a minimal, upstreamable delta: create-snapshots --targets-file/--cardano-db-dir, runtime --era-history-file/--global-parameters-file, the testnet tvar era-history sidecar fix, short-epoch saturate/forecast/consensus guards).
Verified: full nix build-gate green; live epoch-120 short-epoch follow cluster PASS - amaru follows across epochs 3->9 (slot 1088) with zero fatal errors.