This repo is the central location for submitting new results to the GPTNT Leaderboard.
Each PR adds one submission bundle under submissions/. Its folder name is
submissions/YYYYMMDD_<display-slug>_<capfp8>_<suite>_<ver>/. One model
measured across several suites creates several bundles and therefore several
PRs. See CONTRIBUTING.md for the full layout and the
gptnt submission new to submit flow.
Merging a bundle runs the leaderboard publication workflow. It checks out this repository afresh,
builds the active website artifact from multi-self-async and multi-self-sync suite revisions
2 or later, and opens or updates one pull request in GPTNT/gptnt.github.io only when the JSON
changes. It never reads a maintainer's local submissions directory.
Before enabling the workflow, configure these repository settings:
GPTNT_LEADERBOARD_RELEASE_REFvariable: an immutable released GPTNT tag or commit that containsgptnt leaderboard build.WEBSITE_PR_TOKENsecret: a least-privilege token with contents and pull-request write access toGPTNT/gptnt.github.io.- Configure GitHub notifications for this repository and
GPTNT/gptnt.github.ioto receive pull request and Actions updates. The publication workflow does not use an external email provider.
This method of tracking submissions was highly-inspired by ProgramBench, which has their process outlined in ProgramBench/submissions.