fix(#10): post-merge docs/repro fixup — addresses 5 reviewer items#18
Open
hitome0123 wants to merge 1 commit into
Open
fix(#10): post-merge docs/repro fixup — addresses 5 reviewer items#18hitome0123 wants to merge 1 commit into
hitome0123 wants to merge 1 commit into
Conversation
…es 5 reviewer items PR billion-token-one-task#10 was merged but the post-merge review (Protocol-zero-0, 2026-05-16) flagged 5 cleanup items. This commit closes all 5, plus a couple of adversarial-review nits surfaced by re-running the recipe ourselves. 1. Missing scripts/demo_agenda_loop.py referenced in the PR repro recipe. Added a thin wrapper that delegates to build_agenda_loop_acceptance.main(); inserts repo root onto sys.path so the documented one-liner `python scripts/demo_agenda_loop.py` works (without -m). 2. Stale `commit: ef15797` in all 6 acceptance bundles vs HEAD 44ae888. Re-ran every builder; added scripts/build_manuscript_venue_routing_umbrella.py so the umbrella JSON (issue billion-token-one-task#11) is regenerated deterministically from sub-bundles + git HEAD instead of being hand-written. 3. `agenda_loop_acceptance.json` claimed byte-reproducible but the underlying packet contained wallclock-derived fields. Added _strip_non_deterministic() that drops latency_ms_*, latency_speedup_x, peak_memory_mb, DB ids, and abs paths before hashing; renamed the JSON field result_packet_sha256 -> result_packet_structural_sha256 so the claim is honest. Also applied the same strip to `experiment.delta` so the bundle JSON itself is byte-stable across runs. 4. README + docs referenced agents/venues.yaml (wrong path) and stale venue names neurips2025/emnlp2025/iccv2025. Fixed to manuscript_venues/venues_v1.yaml and the actual 6 registered venues: iclr2026, neurips2024, icml2024, acl_arr, cvpr2024, arxiv_plain. 5. New scripts/verify_acceptance.sh — one-liner that wipes scratch DBs, runs all 5 builders + the umbrella, parses each artifact JSON, cross-checks the commit field against `git rev-parse HEAD`, and prints `PASS — 6/6 acceptance bundles regenerated at commit <sha>` only when everything succeeds. Verification: $ bash scripts/verify_acceptance.sh PASS — 6/6 acceptance bundles regenerated at commit 44ae888 $ python3 -m pytest tests/test_top_venue_adapters.py \ tests/test_venue_router.py tests/test_format_linter.py \ tests/test_manuscript_routes.py tests/test_template_adapter.py \ tests/test_venue_router_tiebreak.py -q 50 passed, 6 subtests passed in 0.65s $ python scripts/demo_agenda_loop.py # directly, no -m [ok] wrote artifacts/agenda_loop_acceptance.json Two runs of build_agenda_loop_acceptance now produce identical result_packet_structural_sha256 (087c1d13...) and identical experiment.delta {approximation_l2_distance, relative_error}. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
|
1 similar comment
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to merged PR #10. Addresses all 5 cleanup items from Protocol-zero-0's post-merge review (2026-05-16) plus a couple of adversarial nits we caught on second pass.
Reviewer's note: "第 5 条做完,前 4 条会顺带暴露并修掉" — building
verify_acceptance.shindeed surfaced the stale commit fields and the non-deterministic delta, so this PR closes everything in one pass.Reviewer checklist
scripts/demo_agenda_loop.py— new thin wrapper that delegates tobuild_agenda_loop_acceptance.main(). Inserts repo root ontosys.pathso the documented one-linerpython scripts/demo_agenda_loop.pyworks directly (not just with-m).commit: ef15797in all 6 acceptance bundles — re-ran every builder; all 6 now carrycommit: 44ae888a0c1c…. Addedscripts/build_manuscript_venue_routing_umbrella.pyso the umbrella JSON (issue [Epic] Manuscript Venue Routing + Multi-Template Pipeline (会议路由 + 多模板论文管线) #11) is regenerated deterministically from sub-bundles +git rev-parse HEADinstead of being hand-written.result_packet_sha256— added_strip_non_deterministic()covering wallclock-derived metrics (latency_ms_*,peak_memory_mb,latency_speedup_x), DB auto-increment ids, and abs paths. Renamed the JSON field toresult_packet_structural_sha256so the claim is honest. Also applied the same strip toexperiment.deltaso the bundle JSON itself is byte-stable across runs (was still leakinglatency_speedup_xafter the rename — caught on second pass).agents/venues.yaml→manuscript_venues/venues_v1.yaml+ stale venue names — fixed every reference inREADME.md,docs/top_venue_manuscript_chain.md, and the diagram block. The 6 actually registered venues are now consistently named everywhere:iclr2026, neurips2024, icml2024, acl_arr, cvpr2024, arxiv_plain. `grep -rn 'agents/venues.yaml'` returns 0 hits; `grep -rn 'neurips2025|emnlp2025|iccv2025'` returns 0 hits.scripts/verify_acceptance.sh— one-liner that any contributor can run from a clean venv. Wipes scratch DBs, runs all 5 builders + the umbrella, parses each artifact JSON, cross-checks thecommitfield againstgit rev-parse HEAD, and printsPASS — 6/6 acceptance bundles regenerated at commit <sha>only when everything succeeds. Usesset -Eeuo pipefailso any builder failure short-circuits with a clear message.Verification (re-run on the head commit of this PR)
```
$ bash scripts/verify_acceptance.sh
[verify] regenerating d1 template router (python -m scripts.build_d1_acceptance)
[verify] regenerating d2 top-venue adapters (python -m scripts.build_d2_acceptance)
[verify] regenerating d3 format linter (python -m scripts.build_d3_acceptance)
[verify] regenerating d4 manuscript routing (python -m scripts.build_d4_acceptance)
[verify] regenerating agenda loop (#9) (python -m scripts.build_agenda_loop_acceptance)
[verify] regenerating umbrella (#11) (python -m scripts.build_manuscript_venue_routing_umbrella)
[verify] verifying 6 acceptance bundles exist + parse + match HEAD
ok artifacts/agenda_loop_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d1_template_router_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d2_top_venue_adapters_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d3_format_linter_acceptance.json (commit=44ae888a0c1c)
ok artifacts/d4_manuscript_routing_api_acceptance.json (commit=44ae888a0c1c)
ok artifacts/manuscript_venue_routing_acceptance.json (commit=44ae888a0c1c)
================================================================
PASS — 6/6 acceptance bundles regenerated at commit 44ae888
```
Epic-scope tests (touched paths only):
```
$ python3 -m pytest tests/test_top_venue_adapters.py tests/test_venue_router.py \
tests/test_format_linter.py tests/test_manuscript_routes.py \
tests/test_template_adapter.py tests/test_venue_router_tiebreak.py -q
50 passed, 6 subtests passed in 0.65s
```
Reproducibility proof (two consecutive runs of the agenda loop builder):
```
run1.experiment.delta = {"approximation_l2_distance": 10.13387978273153, "relative_error": 0.7670742721257483}
run2.experiment.delta = {"approximation_l2_distance": 10.13387978273153, "relative_error": 0.7670742721257483}
run1.result_packet_structural_sha256 = 087c1d13348aefbad55a921f3380984de3719cfc581ff300fbe6adf6ab6f67e8
run2.result_packet_structural_sha256 = 087c1d13348aefbad55a921f3380984de3719cfc581ff300fbe6adf6ab6f67e8
```
Documented one-liner now works directly (no
-mneeded):```
$ python scripts/demo_agenda_loop.py
[ok] wrote artifacts/agenda_loop_acceptance.json
selection_id=1 run_id=2 gate=block bundle_id=None review_id=1 plan_id=1
```
Files
New (3)
Modified (11)
Test plan
Related: #9, #10, #11, #12, #13, #14, #15.
🤖 Generated with Claude Code