Reproducing scores on the posttrainbench website and making submissions

[wise-east]

Can you share the model weights of the agent-trained models on huggingface for reproducibility of the results on the leaderboard?
Is there an official way to submit a model to be recognized on the posttrainbench leaderboard?

[hrdkbhatnagar]

Hey Justin!

We have shared the agent traces at https://huggingface.co/datasets/aisa-group/PostTrainBench-Trajectories for transparency, but have decided not to upload the model weights too, as it would cost us a lot in storage (even on HF).

Some napkin math:

We have 84 runs (7 tasks x 4 base models x 3 runs) for each agent, and the model weights are roughly around 5 GB each, this would mean for each agent we need ~420 GB. And since we have around 22 agents, this would be around 9 TB in total.

Regarding a way to submit a model, yes that's definitely possible. If you could upload the files somewhere (including the agent traces and the judge's output and final model weights) then we could verify it ourself. Feel free to also email us if you need more info on this!

[wise-east]

thanks!