Skip to content

Integrate post-solve hypergraph region optimizer - #2100

Closed
seveibar wants to merge 24 commits into
mainfrom
agent/benchmark-unravel-region-optimizer
Closed

Integrate post-solve hypergraph region optimizer#2100
seveibar wants to merge 24 commits into
mainfrom
agent/benchmark-unravel-region-optimizer

Conversation

@seveibar

@seveibar seveibar commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • pin tiny-hypergraph#166 at 8b4a28e and run its post-solve optimizer on every solved mutable graph
  • preserve caller cost factors, exhaust topology-preserving boundary untwists, and reserve whole-route A* for routes whose exact removal can lower the current bottleneck
  • keep preloaded routes immutable per route while retaining them as congestion, collision, and boundary-order anchors
  • remove the routed-trace occupancy heuristic and benchmark-shaped alternative scoring path; the downstream probability model remains uniform
  • pin high-density-repair03#71 at 1dfd623 for exact physical-pad identity, tangent detours, via-aware doglegs, same-net via canonicalization, and faster candidate DRC
  • restore original physical pad geometry after point-pair preprocessing splits rotated pads into routing fragments
  • preserve synthetic movable-preload connection topology through post-regional repair so exact DRC errors map to the correct routes
  • reject speculative shared-root stitch paths containing a third physical terminal, while keeping final stitch identity validation strict
  • iterate repair only on strict global exact-DRC improvement, with route-state cycle detection
  • continue post-regional exact repair through via-only residues instead of stopping when trace/pad errors are exhausted

No benchmark-shaped control flow

There is no sample ID, dataset classifier, density activation threshold, route-count gate, target-reduction cutoff, alternate pipeline portfolio, or acceptance ratio. The optimizer always runs after a successful graph solve and its output is consumed directly. Fixed-vs-movable behavior depends only on supplied immutable copper.

The performance work preserves every candidate budget, portfolio phase, branch order, and exact-DRC acceptance rule. It accelerates the shared evaluator rather than skipping work.

Region-cost result

./benchmark.sh in tiny-hypergraph#166:

Metric Before After Reduction
P50 max region cost 1.440 0.595 58.7%
P95 max region cost 3.464 1.669 51.8%

All 8 cases complete, preserve every route, and improve. Total wall time is 37.835s; average optimizer time is 2.541s.

Downstream engineering evidence

SRJ18 sample 2 isolated the dense-routing relationship the aggregate benchmark had hidden. Compared on the same local machine:

Metric Main f822ca1 PR Change
high-density iterations 1,116,572 869,438 -22.1%
physical errors before exact repair 141 35 -75.2%
physical errors after exact repair 52 7 -86.5%
time before exact repair 103.403s 99.449s -3.8%
exact repair 32.951s 29.164s -11.5%
total through exact repair 136.354s 128.613s -5.7%

The region optimizer lowers downstream routing failure probability by 4.46% on this sample and reduces the detailed router's actual iteration count. Profiling the earlier timeout showed that improved routes gave exact repair many useful candidates, while the evaluator rebuilt a string-keyed spatial grid for each one.

In high-density-repair03#71, numeric spatial columns, stable item IDs, typed generation marking, and immutable connectivity memoization remove that repeated allocation without changing candidate ordering or error semantics. Unprofiled exact repair falls from 51.009s to 29.164s (42.8%) with the same 80 candidate attempts and identical branch decisions/error identities.

SRJ18 sample 13 exposed the optimizer's own contention cost. Its 5.53M exact A* expansions performed 125.5M marginal chord lookups, and the old cache discarded scores after every route-removed state. tiny-hypergraph@8b4a28e uses region-cache object identity, compact region-local port indexes, and paged typed-array rows; owner-independent crossing counts persist across searches while same-net corrections remain exact.

An immediate old/new pair on the same local machine produced:

Metric Previous cache Shared chord memo Change
replacement-cost cache misses 59.48M 10.29M -82.7%
optimizer time 22.63s 14.07s -37.8%
port-point phase 32.62s 23.63s -27.5%
total through exact DRC 121.16s 115.31s -4.8%

Both runs performed 5,533,365 reroute iterations, accepted the same 21 mutations, produced the same region costs and downstream-risk reduction, ran 756,186 high-density iterations, and ended with the same exact-DRC counts. The new CPU profile puts the old intersection scanner below 1% of optimizer time.

The next profile found an independent quadratic handoff cost: Pipeline 7 requested one node probability at a time, and each request rematerialized every solved tiny-hypergraph node. On sample 6, one high-density constructor step spent 66 seconds doing that repeated reconstruction and advanced the real router by zero iterations. The solver now computes the unchanged per-node scores from one materialization and reuses them. This preserves preloaded-route filtering semantics and changes no search budget, branch, ordering, or acceptance rule.

With that cache, a 60-second sample 6 window advances from 0 iterations/0 routes to 434,200 iterations/976 routes. The full production benchmark case now completes locally in 284.4s instead of timing out at 360s, after 1,212,713 high-density iterations. Sample 13 improves again from 115.31s to 98.86s through exact DRC (-14.3%) with identical optimizer iterations/mutations, high-density iterations, and 156 -> 98 exact DRC result.

The final fresh same-machine GitHub comparison improves aggregate routing outcomes on one sequentially shared 8-vCPU worker:

Metric Main f822ca1 PR 57a3232 Change
completion 75.0% 81.3% +6.3 pp
relaxed DRC pass 43.8% 50.0% +6.2 pp
timeouts 4 3 -1
P50 time 139.4s 125.2s -10.2%
P60 time 202.0s 186.5s -7.7%
P80 time 360.0s 319.6s -11.2%

That run has two improved outcomes and zero regressions: sample 2 changes from timeout to solved, and sample 9 changes from solved-with-DRC-errors to DRC-passing. Across the 12 samples completed by both revisions, the PR is faster on 8, lowers combined runtime by 3.1%, lowers relaxed-DRC errors from 146 to 116 (-20.5%), and lowers average vias from 181.83 to 178.25. The bot's all-completed average includes sample 2's 465 vias only on the PR because main timed out.

The production optimizer reports lower total region cost and lower downstream failure risk on all 13 completed PR samples; max region cost improves on 9 and is unchanged on 4, with no increase. Aggregate total region cost falls 291.81 -> 257.42 (-11.8%), and the median downstream-risk reduction is 3.02%. See workflow 31841758356.

SRJ18 sample 3 exposed a separate correctness issue. Point-pair preprocessing split rotated pcb_smtpad_56 into three axis-aligned fragments with the same physical ID; the candidate evaluator deduplicated those fragments before restoring the original shape and missed a real 0.0601 mm pad/trace clearance. Pipeline 7 now reconstructs one original physical obstacle by exact metadata identity, unions its processed connectivity aliases, and repairs the sample to zero relaxed DRC errors.

Other profiled fixes remain dependency- and geometry-driven:

  • sample 12 optimizer time: 41.396s → 6.483s; whole-route searches: 862 → 13; A* expansions: 7.85M → bounded bottleneck-causal searches
  • SRJ20 sample169: strict max/total region-cost reduction, zero exact DRC errors, at most 33 vias
  • SRJ23 sample46: via-only exact residue now participates in the same strict improvement loop and finishes with zero relaxed DRC errors

Validation

  • installed both pinned Git SHAs; no local links
  • tiny-hypergraph: 143 tests pass; format/type/test CI green
  • high-density-repair03: 69 tests pass; format/type/test CI green
  • Pipeline 7 evaluators, preloaded-route compatibility, node-score cache, and SRJ18 sample 3 integration: 8 passed, 0 failed
  • bunx tsc --noEmit
  • git diff --check
  • autorouter build, format, typecheck, ID-hack guard, and Vercel checks pass
  • all 31 changed Linux SVG baselines are copied from their exact CI received artifacts and normalized without visual changes

The requested /benchmark --dataset 18 --same-machine completed successfully on code head 57a3232c in workflow 31841758356. The later commits only preserve preloaded-route score semantics and refresh snapshots; SRJ18 execution is unchanged.

@vercel

vercel Bot commented Aug 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
capacity-node-autorouter Ready Ready Preview Aug 14, 2026 9:24pm

Request Review

@tscircuitbot

Copy link
Copy Markdown
Contributor

Benchmark This PR

Run benchmarks by commenting on this PR:

/benchmark [benchmark.sh args...]
/benchmark-long [benchmark.sh args...]
/benchmark-all
/benchmark --same-machine
/benchmark-all --same-machine

Comment /benchmark to run the default dataset, or append any arguments accepted by ./benchmark.sh.
Comment /benchmark-long for an 8-vCPU run that defaults to 8 workers and has an eight-hour timeout.
Comment exactly /benchmark-all to start separate workflow runs and result comments for the default dataset plus srj18, srj19, srj20, srj21, and srj23.
Append --same-machine to /benchmark or /benchmark-all to compare current main and the PR head sequentially on the same 8-vCPU Blacksmith runner.

Everything after /benchmark or /benchmark-long is safely forwarded to ./benchmark.sh, except --profile-solvers, which enables profile comparison tables, and --same-machine, which enables the paired runner.
Examples: /benchmark --dataset 18 --sample-timeout 2000s, /benchmark --pipeline 7 --scenario-limit 20, /benchmark all 20 --concurrency 8, /benchmark --same-machine, /benchmark-all --same-machine, and /benchmark-long --dataset 18.

Use /update-snapshots (or /us) to run BUN_UPDATE_SNAPSHOTS=1 bun test --timeout 120_000 on the PR branch and auto-commit snapshot updates.
Use /usf to read recent failed test files, update and verify their exact CI test shards, and auto-commit only their snapshots. It uses the configured fast benchmark runner by default; use /usf --ubuntu-latest for GitHub-hosted x64 CI parity.

Any PR whose title contains [BENCHMARK TEST] will automatically run one default-dataset benchmark on PR updates; it does not post a PR result comment.

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01kzyebsrbg8xwz2dxqtsabwaj-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: 629e47c · PR: a0c31fb

Solver Metric Main PR Delta
Pipeline7 Completion 56.3% (🕒43.8%) 56.3% (🕒43.8%) 0.0 pp
Pipeline7 Relaxed DRC pass 31.3% (🕒43.8%) 25.0% (🕒43.8%) -6.3 pp
Pipeline7 Timeouts 7 7 0
Pipeline7 P50 time 246.5s 280.4s +13.8%
Pipeline7 P60 time 360.0s 360.0s 0.0%
Pipeline7 P70 time 360.0s 360.0s 0.0%
Pipeline7 P80 time 360.0s 360.0s 0.0%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 166.22 148.89 -10.4%

Outcome changes: 1 improved, 2 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (3)
Solver Sample Main PR Main time PR time Delta
Pipeline7 1 DRC passed Solved (DRC failed) 72.7s 82.4s Regressed
Pipeline7 4 Timeout Solved (DRC failed) 360.0s 337.9s Improved
Pipeline7 8 Solved (DRC failed) Timeout 301.7s 360.0s Regressed

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01kzyk2zxjctkwqqnnz1rcq09k-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: 2a5539f · PR: 61eee3c

Solver Metric Main PR Delta
Pipeline7 Completion 68.8% (🕒31.3%) 50.0% (🕒50.0%) -18.8 pp
Pipeline7 Relaxed DRC pass 37.5% (🕒31.3%) 31.3% (🕒50.0%) -6.2 pp
Pipeline7 Timeouts 5 8 +3
Pipeline7 P50 time 184.6s 307.7s +66.6%
Pipeline7 P60 time 255.5s 360.0s +40.9%
Pipeline7 P70 time 335.0s 360.0s +7.5%
Pipeline7 P80 time 360.0s 360.0s 0.0%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 184.00 148.75 -19.2%

Outcome changes: 0 improved, 3 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (3)
Solver Sample Main PR Main time PR time Delta
Pipeline7 8 Solved (DRC failed) Timeout 236.8s 360.0s Regressed
Pipeline7 12 DRC passed Timeout 309.9s 360.0s Regressed
Pipeline7 13 Solved (DRC failed) Timeout 255.5s 360.0s Regressed

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

The paired benchmark ended with cancelled before both reports were produced.

Workflow: View run
Artifact: Download raw reports

…el-region-optimizer

# Conflicts:
#	package.json
@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

The paired benchmark ended with cancelled before both reports were produced.

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01kzyp8vc7qenk7wdakznrn5c1-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: 2a5539f · PR: 17e04f5

Solver Metric Main PR Delta
Pipeline7 Completion 68.8% (🕒31.3%) 62.5% (🕒37.5%) -6.3 pp
Pipeline7 Relaxed DRC pass 37.5% (🕒31.3%) 37.5% (🕒37.5%) 0.0 pp
Pipeline7 Timeouts 5 6 +1
Pipeline7 P50 time 174.6s 187.0s +7.1%
Pipeline7 P60 time 244.1s 304.3s +24.7%
Pipeline7 P70 time 329.0s 360.0s +9.4%
Pipeline7 P80 time 360.0s 360.0s 0.0%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 184.00 180.10 -2.1%

Outcome changes: 0 improved, 1 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (1)
Solver Sample Main PR Main time PR time Delta
Pipeline7 13 Solved (DRC failed) Timeout 244.1s 360.0s Regressed

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01m00srh7wjfj0a0msyx7wzg1y-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: f822ca1 · PR: accd698

Solver Metric Main PR Delta
Pipeline7 Completion 81.3% (🕒18.8%) 75.0% (🕒25.0%) -6.3 pp
Pipeline7 Relaxed DRC pass 43.8% (🕒18.8%) 37.5% (🕒25.0%) -6.3 pp
Pipeline7 Timeouts 3 4 +1
Pipeline7 P50 time 144.0s 155.9s +8.2%
Pipeline7 P60 time 211.3s 231.7s +9.7%
Pipeline7 P70 time 242.7s 331.6s +36.6%
Pipeline7 P80 time 354.2s 360.0s +1.6%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 210.15 178.42 -15.1%

Outcome changes: 0 improved, 2 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (2)
Solver Sample Main PR Main time PR time Delta
Pipeline7 2 Solved (DRC failed) Timeout 354.2s 360.0s Regressed
Pipeline7 3 DRC passed Solved (DRC failed) 44.6s 41.9s Regressed

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/usf

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

📸 Update Snapshots (focused)

❌ Snapshot update failed. Please inspect workflow logs.

Focused test files:

  • tests/autorouting-dataset18-sample10-exact-drc-residue-visual.test.ts
  • tests/bugs/bugreport18-1b2d06.test.ts
  • tests/bugs/bugreport44-0ec411.test.ts
  • tests/bugs/bugreport47-8ee80e-esp32-breakout.test.ts
  • tests/bugs/bugreport48-569cfe.test.ts
  • tests/bugs/bugreport51-7db9f8.test.ts
  • tests/bugs/bugreport52-a9573e.test.ts
  • tests/bugs/bugreport57-51db46.test.ts
  • tests/bugs/bugreport58-b69d72.test.ts
  • tests/bugs/bugreport59-82431e.test.ts
  • tests/bugs/bugreport62-0f6ca4.test.ts
  • tests/bugs/bugreport63-274be2.test.ts
  • tests/bugs/bugreport64-be7d8f.test.ts
  • tests/bugs/bugreport66-e6c7e0.test.ts
  • tests/bugs/bugreport68-41562e.test.ts
  • tests/bugs/bugreport68-solar-battery-charger.test.ts
  • tests/bugs/bugreport71-dd7d15.test.ts
  • tests/bugs/bugreport73-qfp16-pipeline4-visual.test.ts
  • tests/bugs/bugreport76-373c24.test.ts
  • tests/bugs/bugreport77-07f6a7.test.ts
  • ...and 12 more
    CI test shards: 1, 2, 3, 4, 5, 6, 7, 8

Source failure run: https://github.com/tscircuit/tscircuit-autorouter/actions/runs/31830484294

Changed snapshots:

  • tests/features/__snapshots__/pipeline7-full-pipeline-svg-frames.snap.svg

Environment: benchmark runner (ARM64), Bun 1.3.8

🔗 Workflow: View run

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01m00xf4d2kbqmr9kvp5q37hm6-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: f822ca1 · PR: aa4d7c8

Solver Metric Main PR Delta
Pipeline7 Completion 68.8% (🕒31.3%) 62.5% (🕒37.5%) -6.3 pp
Pipeline7 Relaxed DRC pass 37.5% (🕒31.3%) 43.8% (🕒37.5%) +6.3 pp
Pipeline7 Timeouts 5 6 +1
Pipeline7 P50 time 215.2s 205.1s -4.7%
Pipeline7 P60 time 288.4s 342.6s +18.8%
Pipeline7 P70 time 330.0s 360.0s +9.1%
Pipeline7 P80 time 360.0s 360.0s 0.0%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 169.91 162.10 -4.6%

Outcome changes: 1 improved, 1 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (2)
Solver Sample Main PR Main time PR time Delta
Pipeline7 9 Solved (DRC failed) DRC passed 137.8s 129.5s Improved
Pipeline7 13 Solved (DRC failed) Timeout 288.4s 360.0s Regressed

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/usf

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

📸 Update Snapshots (focused)

❌ Snapshot update failed. Please inspect workflow logs.

Focused test files:

  • tests/autorouting-dataset18-sample10-exact-drc-residue-visual.test.ts
  • tests/bugs/bugreport18-1b2d06.test.ts
  • tests/bugs/bugreport44-0ec411.test.ts
  • tests/bugs/bugreport47-8ee80e-esp32-breakout.test.ts
  • tests/bugs/bugreport48-569cfe.test.ts
  • tests/bugs/bugreport51-7db9f8.test.ts
  • tests/bugs/bugreport52-a9573e.test.ts
  • tests/bugs/bugreport57-51db46.test.ts
  • tests/bugs/bugreport58-b69d72.test.ts
  • tests/bugs/bugreport59-82431e.test.ts
  • tests/bugs/bugreport62-0f6ca4.test.ts
  • tests/bugs/bugreport63-274be2.test.ts
  • tests/bugs/bugreport64-be7d8f.test.ts
  • tests/bugs/bugreport66-e6c7e0.test.ts
  • tests/bugs/bugreport68-41562e.test.ts
  • tests/bugs/bugreport68-solar-battery-charger.test.ts
  • tests/bugs/bugreport71-dd7d15.test.ts
  • tests/bugs/bugreport73-qfp16-pipeline4-visual.test.ts
  • tests/bugs/bugreport76-373c24.test.ts
  • tests/bugs/bugreport77-07f6a7.test.ts
  • ...and 11 more
    CI test shards: 1, 2, 3, 4, 5, 6, 7, 8

Source failure run: https://github.com/tscircuit/tscircuit-autorouter/actions/runs/31835393537

Changed snapshots:

  • tests/__snapshots__/autorouting-dataset18-sample10-exact-drc-residue-visual-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport18-1b2d06.snap.svg
  • tests/bugs/__snapshots__/bugreport44-0ec411.snap.svg
  • tests/bugs/__snapshots__/bugreport47-8ee80e-esp32-breakout.snap.svg
  • tests/bugs/__snapshots__/bugreport48-569cfe.snap.svg
  • tests/bugs/__snapshots__/bugreport51-7db9f8-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport52-a9573e.snap.svg
  • tests/bugs/__snapshots__/bugreport58-b69d72-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport59-82431e-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport62-0f6ca4.snap.svg
  • tests/bugs/__snapshots__/bugreport64-be7d8f.snap.svg
  • tests/bugs/__snapshots__/bugreport66-e6c7e0.snap.svg
  • tests/bugs/__snapshots__/bugreport68-41562e.snap.svg
  • tests/bugs/__snapshots__/bugreport68-solar-battery-charger.snap.svg
  • tests/bugs/__snapshots__/bugreport71-dd7d15-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport73-qfp16-pipeline4-visual.snap.svg
  • tests/bugs/__snapshots__/bugreport76-373c24.snap.svg
  • tests/bugs/__snapshots__/bugreport77-07f6a7.snap.svg
  • tests/bugs/__snapshots__/bugreport80-75ab58-linux.snap.svg
  • tests/bugs/__snapshots__/bugreport82-0e99ec.snap.svg
  • tests/bugs/__snapshots__/bugreport84-726193.snap.svg
  • tests/bugs/__snapshots__/bugreport87-76a6de.snap.svg
  • tests/bugs/__snapshots__/bugreport88-9a86ed.snap.svg
  • tests/bugs/__snapshots__/bugreport90-6dc620.snap.svg
  • tests/features/__snapshots__/pipeline4-dataset01-circuit011-visual.snap.svg
  • tests/features/__snapshots__/pipeline4-dataset01-circuit015-visual.snap.svg
  • tests/features/__snapshots__/same-net-via-clearance-repair-visual.snap.svg
  • tests/repro/__snapshots__/repro117-standalone-simple-route.snap.svg

Environment: benchmark runner (ARM64), Bun 1.3.8

🔗 Workflow: View run

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01m010z9c977b1jndjsqp7cm5s-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: f822ca1 · PR: 483a65a

Solver Metric Main PR Delta
Pipeline7 Completion 75.0% (🕒25.0%) 81.3% (🕒18.8%) +6.3 pp
Pipeline7 Relaxed DRC pass 43.8% (🕒25.0%) 50.0% (🕒18.8%) +6.2 pp
Pipeline7 Timeouts 4 3 -1
Pipeline7 P50 time 146.7s 139.6s -4.9%
Pipeline7 P60 time 209.8s 225.2s +7.4%
Pipeline7 P70 time 254.3s 294.3s +15.7%
Pipeline7 P80 time 360.0s 332.4s -7.7%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 181.83 200.31 +10.2%

Outcome changes: 2 improved, 0 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (2)
Solver Sample Main PR Main time PR time Delta
Pipeline7 2 Timeout Solved (DRC failed) 360.0s 332.4s Improved
Pipeline7 9 Solved (DRC failed) DRC passed 82.3s 72.3s Improved

Workflow: View run
Artifact: Download raw reports

@seveibar

Copy link
Copy Markdown
Contributor Author

/benchmark --dataset 18 --same-machine

@tscircuitbot

tscircuitbot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Same Machine Benchmark Results

Both revisions ran sequentially in one Blacksmith job on blacksmith-01m0125qsp79xkdj2n7d2ta6ns-8vcpu.

Dataset: srj18 · Scenarios: 16
Main: f822ca1 · PR: 57a3232

Solver Metric Main PR Delta
Pipeline7 Completion 75.0% (🕒25.0%) 81.3% (🕒18.8%) +6.3 pp
Pipeline7 Relaxed DRC pass 43.8% (🕒25.0%) 50.0% (🕒18.8%) +6.2 pp
Pipeline7 Timeouts 4 3 -1
Pipeline7 P50 time 139.4s 125.2s -10.2%
Pipeline7 P60 time 202.0s 186.5s -7.7%
Pipeline7 P70 time 232.0s 250.2s +7.8%
Pipeline7 P80 time 360.0s 319.6s -11.2%
Pipeline7 P90 time 360.0s 360.0s 0.0%
Pipeline7 P95 time 360.0s 360.0s 0.0%
Pipeline7 Average vias 181.83 200.31 +10.2%

Outcome changes: 2 improved, 0 regressed. Timing percentiles include solved and timed-out samples; negative timing deltas are faster.

Changed outcomes (2)
Solver Sample Main PR Main time PR time Delta
Pipeline7 2 Timeout Solved (DRC failed) 360.0s 319.6s Improved
Pipeline7 9 Solved (DRC failed) DRC passed 89.7s 58.9s Improved

Workflow: View run
Artifact: Download raw reports

@github-actions

Copy link
Copy Markdown

This PR has been automatically marked as stale because it has had no recent activity. It will be closed if no further activity occurs.

@github-actions

Copy link
Copy Markdown

This PR was closed because it has been inactive for 1 day since being marked as stale.

@github-actions github-actions Bot closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants