Skip to content

Commit 8537a0e

Browse files
committed
feat(benchmark): hardware fingerprint, Kaggle notebook, v0.3.0rc1
Make every benchmark result self-describing and add a single-paste Kaggle entry point so the standardised paper sweeps can run unattended on a free T4 / P100 session. swarmtorch/benchmark/hardware.py hardware_info() collects hostname, OS, CPU model, logical cores, RAM, Python + PyTorch + CUDA versions, GPU name + memory + compute capability, and a git short SHA so results are tied to a code state. Cached, free to call repeatedly. print_banner() emits the same fields as a loud block at the top of any script -- so Kaggle / Colab logs open with unambiguous machine identification. swarmtorch/benchmark/run.py run_one() now stamps meta.hardware on every RunResult JSON, so when results are shared across collaborators the runtime is fully recoverable from the file. swarmtorch/benchmark/gpu_vs_numpy.py Same hardware stamp on every GPUBenchResult. report.md opens with a Machine: / GPU: / PyTorch: header line. swarmtorch/benchmark/report.py The aggregated report pulls hardware info from the first result (the producing machine), not the machine rebuilding the report, so summarize_results works correctly across machines. scripts/cloud_bench.py --all flag runs the full standardised suite (ablations + HPO + synthetic + GPU + training). Prints the hardware banner once at the top of the run. bench-training added to the dispatch table. scripts/kaggle_run_all.py Single-paste Kaggle entry point: clone + install + identify + smoke + full --all suite + zip bundle. Designed for the one- line "!curl ... | python" Kaggle workflow. notebooks/kaggle_full_benchmark.ipynb Cell-by-cell version of the same suite, so the user can read each report.md inline as it lands instead of waiting for the whole suite. Default synthetic grid bumped to include d=1000 (the CEC standard ceiling) so the dimensionality-wall claim survives review. pyproject.toml: version 0.2.1 -> 0.3.0rc1 BENCHMARKS.md: Kaggle TL;DR added at top. tests/test_benchmark_run.py: updated meta assertion to allow the new auto-injected hardware field. Test suite: 94 passed, 1 skipped (CUDA-gated), 0 failures.
1 parent f2465f7 commit 8537a0e

11 files changed

Lines changed: 837 additions & 10 deletions

File tree

BENCHMARKS.md

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,40 @@
11
# Running the swarmtorch benchmarks
22

33
This file documents how to reproduce the paper's empirical results.
4-
Two execution paths:
4+
5+
## TL;DR — Kaggle (one paste, ~5 hours, fully standardised)
6+
7+
The cleanest way to produce paper-quality results: paste
8+
`scripts/kaggle_run_all.py` into a single Kaggle code cell with a GPU
9+
runtime. Everything runs end-to-end, every result JSON carries a
10+
hardware fingerprint, and the final zip downloads from
11+
`/kaggle/working/`.
12+
13+
1. Kaggle → New Notebook → Settings:
14+
* Accelerator: **GPU T4 x2** (or P100/V100 if you have access)
15+
* Internet: **On**
16+
2. In a single cell, paste the entire contents of
17+
`scripts/kaggle_run_all.py` and run it.
18+
3. Wait ~4–6 hours (well within Kaggle's 12-hour session limit).
19+
4. Download `swarmtorch_results_<timestamp>.zip` from the Output tab.
20+
21+
This produces:
22+
23+
* Stage 4.1 dimensionality-wall study (synthetic functions)
24+
* Stage 4.2 real-NN training (MNIST + CIFAR-10)
25+
* Stage 4.3 HPO comparison (metaheuristics vs Random / TPE / Hyperband)
26+
* Stage 4.4 init-strategy and swarm-size ablations
27+
* Stage 6 swarmtorch GPU vs NumPy headline speedup
28+
29+
Every JSON includes `meta.hardware` with CPU model, GPU model, RAM,
30+
OS, Python and PyTorch versions, so paper figures stay traceable to
31+
the exact runtime that produced them.
32+
33+
## Other paths
534

635
* **Local / Linux with `make` installed** — use the Makefile.
7-
* **Cloud agent (Jules), Colab, Kaggle, slim containers** — use
8-
`python scripts/cloud_bench.py`. No `make` dependency; same sweeps.
36+
* **Cloud agent (Jules), Colab, custom containers** — use
37+
`python scripts/cloud_bench.py` (no `make` dependency).
938

1039
---
1140

0 commit comments

Comments
 (0)