L1 Paper Results & Code Version discrapencies

[odedsc]

Thank you for sharing this repository!

I'm trying to reproduce the results from "L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning".
I'm curious if the reported results were achieved using the code before or after the "Refactor to Latest Versions" commit on May 5th?

I'm asking because when I fine-tuned the L1-Qwen-1.5B-Exact model to L1-Qwen-1.5B-Max using 'run_l1_max.sh', I wasn't able to match the performance described in the paper.
Would that refactor introduce a significant difference in the fine-tuning process?

Could you also share the specific package versions used in your environment? I've encountered several bugs during installation when trying to set up the environment without specified library versions.

Thanks.

[Pranjal2041]

The results were achieved using code before May 5th.

Nonetheless I wouldn't expect a major discrepancy. Can you share, the exact configuration used and the results you achieved?

Regarding specific versions instructions here should have worked. If not can you share specific errors, so we can fix the errors. Thanks!

[odedsc]

Thank you for your response!

It's good to confirm that the results were indeed achieved with the pre-May 5th code, as that's also the version I'm using.

I fine-tuned the L1-Qwen-1.5B-Exact model to L1-Qwen-1.5B-Max on 4x A100 GPUs.
I've consistently observed that the results for global_step_120 are lower than the ones reported in the paper, and I don't believe the number of GPUs is the cause.

Here are the pass@1 and pass@16 results across all relevant datasets from my runs:

model_path,dataset,pass@1,pass@16 (512)
actor/global_step_120,aime2025.parquet,0.03125,0.16666666666666666
actor/global_step_120,math.parquet,0.614125,0.85
actor/global_step_120,amc.parquet,0.36521084337349397,0.6385542168674698
actor/global_step_120,olympiad_bench.parquet,0.23,0.45925925925925926
actor/global_step_120,gpqa.parquet,0.3134469696969697,0.7727272727272727
actor/global_step_120,mmlu_1000.parquet,0.45025,0.856
actor/global_step_120,lsat.parquet,0.20353260869565218,0.6391304347826087
actor/global_step_120,aime.parquet,0.014583333333333334,0.2

model_path,dataset,pass@1,pass@16 (1024)
actor/global_step_120,aime2025.parquet,0.09791666666666667,0.3
actor/global_step_120,math.parquet,0.7385,0.914
actor/global_step_120,amc.parquet,0.48569277108433734,0.7469879518072289
actor/global_step_120,olympiad_bench.parquet,0.3437037037037037,0.56
actor/global_step_120,gpqa.parquet,0.2566287878787879,0.7474747474747475
actor/global_step_120,mmlu_1000.parquet,0.46125,0.857
actor/global_step_120,lsat.parquet,0.23722826086956522,0.6826086956521739
actor/global_step_120,aime.parquet,0.07916666666666666,0.4

model_path,dataset,pass@1,pass@16 (2048)
actor/global_step_120,aime2025.parquet,0.13958333333333334,0.3
actor/global_step_120,math.parquet,0.79575,0.93
actor/global_step_120,amc.parquet,0.5768072289156626,0.8313253012048193
actor/global_step_120,olympiad_bench.parquet,0.40435185185185185,0.5807407407407408
actor/global_step_120,gpqa.parquet,0.3121843434343434,0.7777777777777778
actor/global_step_120,mmlu_1000.parquet,0.475875,0.873
actor/global_step_120,lsat.parquet,0.2519021739130435,0.7260869565217392
actor/global_step_120,aime.parquet,0.17291666666666666,0.4666666666666667

model_path,dataset,pass@1,pass@16 (3600)
actor/global_step_120,aime2025.parquet,0.16458333333333333,0.3333333333333333
actor/global_step_120,math.parquet,0.821875,0.942
actor/global_step_120,amc.parquet,0.6400602409638554,0.891566265060241
actor/global_step_120,olympiad_bench.parquet,0.43194444444444446,0.6133333333333333
actor/global_step_120,gpqa.parquet,0.319760101010101,0.8282828282828283
actor/global_step_120,mmlu_1000.parquet,0.476125,0.875
actor/global_step_120,lsat.parquet,0.2584239130434783,0.7434782608695653
actor/global_step_120,aime.parquet,0.20416666666666666,0.5

Could you please verify that no additional code changes are required to achieve the results reported in the paper?
In addition, what should be the value of alpha in get_delta_score_max or get_delta_score_linear_both when running run_l1_exact.sh?