Skip to content

update tiny-hypergraph for blocker-search benchmarking - #2500

Open
mohan-bee wants to merge 3 commits into
mainfrom
codex/rv1106-owner-mask-benchmark
Open

update tiny-hypergraph for blocker-search benchmarking#2500
mohan-bee wants to merge 3 commits into
mainfrom
codex/rv1106-owner-mask-benchmark

Conversation

@mohan-bee

@mohan-bee mohan-bee commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • Benchmark the blocker-search optimization on srj18 using the same machine.

Before

  • Autorouter uses the existing tiny-hypergraph revision without this optimization.

After

@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
capacity-node-autorouter Ready Ready Preview Sep 10, 2026 6:26am UTC

Request Review

@tscircuitbot

Copy link
Copy Markdown
Contributor

Benchmark This PR

Run benchmarks by commenting on this PR:

/benchmark [benchmark.sh args...]
/benchmark-long [benchmark.sh args...]
/benchmark-all [benchmark.sh args except --dataset]
/benchmark --same-machine
/benchmark-all --same-machine
/profile --dataset 18

Comment /benchmark to run the default dataset, or append any arguments accepted by ./benchmark.sh.
Comment /benchmark-long for an 8-vCPU run that defaults to 8 workers and has an eight-hour timeout.
Comment /benchmark-all to start separate workflow runs and result comments for dataset01 and dataset18. You may append benchmark arguments such as --pipeline 9; --dataset is not accepted because the command selects both datasets.
Append --same-machine to /benchmark or /benchmark-all to compare current main and the PR head sequentially on the same 8-vCPU Blacksmith runner.
Comment /profile --dataset 18 to compare direct Pipeline 7 stage-time percentages between current main and the PR head sequentially on one Blacksmith runner.

Everything after /benchmark, /benchmark-long, or /benchmark-all is safely forwarded to ./benchmark.sh, except --profile-solvers, which enables profile comparison tables, and --same-machine, which enables the paired runner.
Examples: /benchmark --dataset 18 --sample-timeout 2000s, /benchmark --pipeline 10 --dataset 29, /benchmark --pipeline 10 --dataset 29 --sample 1, /benchmark --pipeline 7 --limit 20, /benchmark all 20 --concurrency 8, /benchmark --same-machine, /benchmark-all --pipeline 9 --same-machine, and /benchmark-long --dataset 18.

Use /update-snapshots (or /us) to run BUN_UPDATE_SNAPSHOTS=1 bun test --timeout 120_000 on the PR branch and auto-commit snapshot updates.
Use /usf to read recent failed test files, update and verify their exact CI test shards, and auto-commit only their snapshots. It uses the configured fast benchmark runner by default; use /usf --ubuntu-latest for GitHub-hosted x64 CI parity.

Any PR whose title contains [BENCHMARK TEST] will automatically run one default-dataset benchmark on PR updates; it does not post a PR result comment.

@mohan-bee

Copy link
Copy Markdown
Contributor Author

/benchmark --pipeline 9 --dataset srj18 --same-machine

@tscircuitbot

tscircuitbot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01m23r2vpvvvdnk5p0ja0qan5s-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: 89d92ef · PR: a9ddf08

Solver Metric Main PR Delta
Pipeline9 Completion 81.3% (🕒6.3%) 81.3% (🕒6.3%) 0.0 pp
Pipeline9 Relaxed DRC pass 75.0% (🕒6.3%) 75.0% (🕒6.3%) 0.0 pp
Pipeline9 DRC issues 4 4 0
Pipeline9 Timeouts 1 1 0
Pipeline9 P50 time 93.0s 95.3s +2.4%
Pipeline9 P60 time 125.2s 128.8s +2.9%
Pipeline9 P70 time 198.6s 202.8s +2.1%
Pipeline9 P80 time 265.0s 258.2s -2.5%
Pipeline9 P90 time 307.3s 304.2s -1.0%
Pipeline9 P95 time 335.4s 336.5s +0.3%
Pipeline9 Average vias 211.85 211.85 0.0%

Outcome changes: 0 improved, 0 regressed. DRC issues are totaled across solved samples. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Workflow: View run
Artifact: Download raw reports

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants