Skip to content

[Refactor] Refactor tool transforms #3221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[Refactor] Refactor tool transforms #3221

wants to merge 1 commit into from

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 24, 2025

No description provided.

@pytorch-bot
Copy link

pytorch-bot bot commented Oct 24, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3221

Note: Links to docs will display an error until the docs builds have been completed.

❌ 9 New Failures, 7 Unrelated Failures

As of commit b556d30 with merge base cccfaa6 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 24, 2025
@github-actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 82.9053μs 81.1997μs 12.3153 KOps/s 12.2818 KOps/s $\color{#35bf28}+0.27\%$
test_tensor_to_bytestream_speed[torch.save] 0.1411ms 0.1405ms 7.1150 KOps/s 7.0829 KOps/s $\color{#35bf28}+0.45\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1215s 0.1208s 8.2813 Ops/s 8.0894 Ops/s $\color{#35bf28}+2.37\%$
test_tensor_to_bytestream_speed[numpy] 2.8398μs 2.8353μs 352.6942 KOps/s 353.1508 KOps/s $\color{#d91a1a}-0.13\%$
test_tensor_to_bytestream_speed[safetensors] 41.3925μs 41.1882μs 24.2788 KOps/s 24.2184 KOps/s $\color{#35bf28}+0.25\%$
test_simple 0.5553s 0.5529s 1.8087 Ops/s 1.7344 Ops/s $\color{#35bf28}+4.28\%$
test_transformed 1.2262s 1.1350s 0.8810 Ops/s 0.8800 Ops/s $\color{#35bf28}+0.12\%$
test_serial 1.6752s 1.6712s 0.5984 Ops/s 0.5907 Ops/s $\color{#35bf28}+1.29\%$
test_parallel 1.1697s 1.1120s 0.8993 Ops/s 0.9080 Ops/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-True-True-True-True] 0.2675ms 46.8634μs 21.3386 KOps/s 22.1651 KOps/s $\color{#d91a1a}-3.73\%$
test_step_mdp_speed[True-True-True-True-False] 60.0210μs 25.8254μs 38.7215 KOps/s 39.0355 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-True-False-True] 66.6410μs 26.4370μs 37.8258 KOps/s 39.2492 KOps/s $\color{#d91a1a}-3.63\%$
test_step_mdp_speed[True-True-True-False-False] 45.8810μs 14.3850μs 69.5170 KOps/s 71.3654 KOps/s $\color{#d91a1a}-2.59\%$
test_step_mdp_speed[True-True-False-True-True] 0.1039ms 49.0462μs 20.3890 KOps/s 20.6564 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[True-True-False-True-False] 59.6310μs 28.4197μs 35.1869 KOps/s 35.4712 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-False-False-True] 77.7120μs 29.1884μs 34.2602 KOps/s 35.7629 KOps/s $\color{#d91a1a}-4.20\%$
test_step_mdp_speed[True-True-False-False-False] 51.5210μs 17.2768μs 57.8811 KOps/s 59.3875 KOps/s $\color{#d91a1a}-2.54\%$
test_step_mdp_speed[True-False-True-True-True] 90.4920μs 51.3187μs 19.4861 KOps/s 19.5273 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-True-False] 76.3110μs 31.2614μs 31.9883 KOps/s 32.8101 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[True-False-True-False-True] 71.9610μs 29.3990μs 34.0148 KOps/s 35.1119 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[True-False-True-False-False] 49.8910μs 17.0458μs 58.6655 KOps/s 59.8773 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-False-False-True-True] 0.1185ms 53.9726μs 18.5279 KOps/s 18.5251 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-False-False-True-False] 73.1410μs 33.4690μs 29.8784 KOps/s 29.8152 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-False-False-False-True] 73.8220μs 31.6804μs 31.5653 KOps/s 32.8150 KOps/s $\color{#d91a1a}-3.81\%$
test_step_mdp_speed[True-False-False-False-False] 53.8710μs 19.6150μs 50.9814 KOps/s 51.0004 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-True-True-True-True] 87.4620μs 51.7615μs 19.3194 KOps/s 19.5318 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-True-True-True-False] 65.9610μs 31.2809μs 31.9684 KOps/s 32.1940 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[False-True-True-False-True] 2.1884ms 33.4832μs 29.8657 KOps/s 31.1794 KOps/s $\color{#d91a1a}-4.21\%$
test_step_mdp_speed[False-True-True-False-False] 56.0810μs 18.8037μs 53.1810 KOps/s 53.3199 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[False-True-False-True-True] 88.6710μs 53.9376μs 18.5399 KOps/s 18.4139 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-True-False-True-False] 83.2710μs 33.6508μs 29.7169 KOps/s 29.6142 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-False-False-True] 75.3110μs 35.2063μs 28.4040 KOps/s 29.2089 KOps/s $\color{#d91a1a}-2.76\%$
test_step_mdp_speed[False-True-False-False-False] 71.4420μs 21.7281μs 46.0234 KOps/s 46.7868 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-False-True-True-True] 92.3110μs 56.5763μs 17.6753 KOps/s 17.5712 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-False-True-True-False] 70.7820μs 36.7891μs 27.1820 KOps/s 27.7687 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-True-False-True] 71.8510μs 35.3900μs 28.2566 KOps/s 28.4647 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-True-False-False] 63.6210μs 21.4246μs 46.6753 KOps/s 46.9841 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-False-True-True] 0.1024ms 58.6142μs 17.0607 KOps/s 16.9125 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-False-True-False] 74.9810μs 38.9742μs 25.6580 KOps/s 25.9148 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[False-False-False-False-True] 75.7420μs 37.5451μs 26.6347 KOps/s 27.1049 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-False-False-False-False] 52.8810μs 23.9578μs 41.7401 KOps/s 42.1390 KOps/s $\color{#d91a1a}-0.95\%$
test_values[generalized_advantage_estimate-True-True] 10.6260ms 9.8944ms 101.0670 Ops/s 102.2135 Ops/s $\color{#d91a1a}-1.12\%$
test_values[vec_generalized_advantage_estimate-True-True] 15.4204ms 11.1737ms 89.4961 Ops/s 89.7438 Ops/s $\color{#d91a1a}-0.28\%$
test_values[td0_return_estimate-False-False] 0.2239ms 0.1311ms 7.6278 KOps/s 7.8952 KOps/s $\color{#d91a1a}-3.39\%$
test_values[td1_return_estimate-False-False] 28.7807ms 27.9883ms 35.7292 Ops/s 37.7156 Ops/s $\textbf{\color{#d91a1a}-5.27\%}$
test_values[vec_td1_return_estimate-False-False] 12.3528ms 11.2381ms 88.9828 Ops/s 88.9204 Ops/s $\color{#35bf28}+0.07\%$
test_values[td_lambda_return_estimate-True-False] 43.0312ms 42.0762ms 23.7664 Ops/s 25.2947 Ops/s $\textbf{\color{#d91a1a}-6.04\%}$
test_values[vec_td_lambda_return_estimate-True-False] 11.4949ms 11.1710ms 89.5171 Ops/s 89.6699 Ops/s $\color{#d91a1a}-0.17\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6421ms 8.4789ms 117.9394 Ops/s 118.6168 Ops/s $\color{#d91a1a}-0.57\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7955ms 1.5381ms 650.1502 Ops/s 668.7810 Ops/s $\color{#d91a1a}-2.79\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7968ms 0.4243ms 2.3568 KOps/s 2.3833 KOps/s $\color{#d91a1a}-1.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.3540ms 29.6189ms 33.7622 Ops/s 42.5654 Ops/s $\textbf{\color{#d91a1a}-20.68\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8864ms 1.7225ms 580.5431 Ops/s 581.5839 Ops/s $\color{#d91a1a}-0.18\%$
test_dqn_speed[False-None] 1.5940ms 1.4461ms 691.4979 Ops/s 699.2476 Ops/s $\color{#d91a1a}-1.11\%$
test_dqn_speed[False-backward] 2.3361ms 1.9859ms 503.5507 Ops/s 516.0724 Ops/s $\color{#d91a1a}-2.43\%$
test_dqn_speed[True-None] 0.9041ms 0.5205ms 1.9212 KOps/s 1.9651 KOps/s $\color{#d91a1a}-2.24\%$
test_dqn_speed[True-backward] 1.0067ms 0.9740ms 1.0267 KOps/s 1.0289 KOps/s $\color{#d91a1a}-0.22\%$
test_dqn_speed[reduce-overhead-None] 0.8539ms 0.5042ms 1.9834 KOps/s 1.9962 KOps/s $\color{#d91a1a}-0.64\%$
test_dqn_speed[reduce-overhead-backward] 1.0069ms 0.9449ms 1.0583 KOps/s 1.0536 KOps/s $\color{#35bf28}+0.44\%$
test_ddpg_speed[False-None] 3.2224ms 2.9169ms 342.8256 Ops/s 349.3401 Ops/s $\color{#d91a1a}-1.86\%$
test_ddpg_speed[False-backward] 4.3083ms 4.1457ms 241.2117 Ops/s 245.1888 Ops/s $\color{#d91a1a}-1.62\%$
test_ddpg_speed[True-None] 1.7960ms 1.3727ms 728.4915 Ops/s 724.3457 Ops/s $\color{#35bf28}+0.57\%$
test_ddpg_speed[True-backward] 2.4027ms 2.3458ms 426.2961 Ops/s 424.7153 Ops/s $\color{#35bf28}+0.37\%$
test_ddpg_speed[reduce-overhead-None] 1.7226ms 1.3525ms 739.3814 Ops/s 745.7931 Ops/s $\color{#d91a1a}-0.86\%$
test_ddpg_speed[reduce-overhead-backward] 2.4008ms 2.3330ms 428.6381 Ops/s 431.6132 Ops/s $\color{#d91a1a}-0.69\%$
test_sac_speed[False-None] 8.3607ms 7.9346ms 126.0303 Ops/s 126.8642 Ops/s $\color{#d91a1a}-0.66\%$
test_sac_speed[False-backward] 11.8320ms 11.2376ms 88.9872 Ops/s 89.5914 Ops/s $\color{#d91a1a}-0.67\%$
test_sac_speed[True-None] 3.3420ms 2.0953ms 477.2608 Ops/s 475.8197 Ops/s $\color{#35bf28}+0.30\%$
test_sac_speed[True-backward] 4.2785ms 4.0048ms 249.7016 Ops/s 227.9187 Ops/s $\textbf{\color{#35bf28}+9.56\%}$
test_sac_speed[reduce-overhead-None] 2.4877ms 2.1008ms 476.0042 Ops/s 478.1659 Ops/s $\color{#d91a1a}-0.45\%$
test_sac_speed[reduce-overhead-backward] 4.6027ms 4.0300ms 248.1403 Ops/s 232.8188 Ops/s $\textbf{\color{#35bf28}+6.58\%}$
test_redq_speed[False-None] 11.0347ms 10.3337ms 96.7705 Ops/s 95.2867 Ops/s $\color{#35bf28}+1.56\%$
test_redq_speed[False-backward] 18.6800ms 18.0084ms 55.5297 Ops/s 55.6590 Ops/s $\color{#d91a1a}-0.23\%$
test_redq_speed[True-None] 4.7543ms 4.3964ms 227.4589 Ops/s 231.3713 Ops/s $\color{#d91a1a}-1.69\%$
test_redq_speed[True-backward] 10.4314ms 9.8552ms 101.4691 Ops/s 92.2607 Ops/s $\textbf{\color{#35bf28}+9.98\%}$
test_redq_speed[reduce-overhead-None] 4.7891ms 4.3290ms 231.0018 Ops/s 224.1930 Ops/s $\color{#35bf28}+3.04\%$
test_redq_speed[reduce-overhead-backward] 11.0683ms 10.0431ms 99.5707 Ops/s 100.0264 Ops/s $\color{#d91a1a}-0.46\%$
test_redq_deprec_speed[False-None] 12.0914ms 11.0454ms 90.5354 Ops/s 90.0343 Ops/s $\color{#35bf28}+0.56\%$
test_redq_deprec_speed[False-backward] 16.8620ms 15.9597ms 62.6578 Ops/s 61.2231 Ops/s $\color{#35bf28}+2.34\%$
test_redq_deprec_speed[True-None] 3.7952ms 3.5959ms 278.0926 Ops/s 278.3310 Ops/s $\color{#d91a1a}-0.09\%$
test_redq_deprec_speed[True-backward] 7.9896ms 7.6634ms 130.4897 Ops/s 129.5691 Ops/s $\color{#35bf28}+0.71\%$
test_redq_deprec_speed[reduce-overhead-None] 4.1705ms 3.5700ms 280.1149 Ops/s 276.7969 Ops/s $\color{#35bf28}+1.20\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.1409ms 7.6317ms 131.0321 Ops/s 125.1574 Ops/s $\color{#35bf28}+4.69\%$
test_td3_speed[False-None] 8.0119ms 7.9568ms 125.6785 Ops/s 118.8785 Ops/s $\textbf{\color{#35bf28}+5.72\%}$
test_td3_speed[False-backward] 11.3409ms 10.8923ms 91.8080 Ops/s 91.9257 Ops/s $\color{#d91a1a}-0.13\%$
test_td3_speed[True-None] 1.8383ms 1.7919ms 558.0758 Ops/s 570.3657 Ops/s $\color{#d91a1a}-2.15\%$
test_td3_speed[True-backward] 3.7099ms 3.5724ms 279.9271 Ops/s 263.0457 Ops/s $\textbf{\color{#35bf28}+6.42\%}$
test_td3_speed[reduce-overhead-None] 1.7858ms 1.7382ms 575.3118 Ops/s 576.0046 Ops/s $\color{#d91a1a}-0.12\%$
test_td3_speed[reduce-overhead-backward] 3.6999ms 3.6061ms 277.3083 Ops/s 239.0019 Ops/s $\textbf{\color{#35bf28}+16.03\%}$
test_cql_speed[False-None] 28.9292ms 25.8935ms 38.6197 Ops/s 38.4401 Ops/s $\color{#35bf28}+0.47\%$
test_cql_speed[False-backward] 39.2684ms 35.5050ms 28.1650 Ops/s 27.7527 Ops/s $\color{#35bf28}+1.49\%$
test_cql_speed[True-None] 12.6682ms 12.2629ms 81.5465 Ops/s 80.4799 Ops/s $\color{#35bf28}+1.33\%$
test_cql_speed[True-backward] 19.7844ms 18.5264ms 53.9770 Ops/s 52.6400 Ops/s $\color{#35bf28}+2.54\%$
test_cql_speed[reduce-overhead-None] 12.9068ms 12.4988ms 80.0077 Ops/s 79.2740 Ops/s $\color{#35bf28}+0.93\%$
test_cql_speed[reduce-overhead-backward] 18.8122ms 18.4378ms 54.2364 Ops/s 54.8393 Ops/s $\color{#d91a1a}-1.10\%$
test_a2c_speed[False-None] 5.7953ms 5.4475ms 183.5720 Ops/s 183.0258 Ops/s $\color{#35bf28}+0.30\%$
test_a2c_speed[False-backward] 12.4231ms 11.8987ms 84.0429 Ops/s 83.6938 Ops/s $\color{#35bf28}+0.42\%$
test_a2c_speed[True-None] 4.2087ms 3.7299ms 268.1006 Ops/s 264.8039 Ops/s $\color{#35bf28}+1.24\%$
test_a2c_speed[True-backward] 8.8243ms 8.5973ms 116.3161 Ops/s 102.6209 Ops/s $\textbf{\color{#35bf28}+13.35\%}$
test_a2c_speed[reduce-overhead-None] 3.8537ms 3.7039ms 269.9858 Ops/s 274.2552 Ops/s $\color{#d91a1a}-1.56\%$
test_a2c_speed[reduce-overhead-backward] 8.9742ms 8.8282ms 113.2735 Ops/s 111.2780 Ops/s $\color{#35bf28}+1.79\%$
test_ppo_speed[False-None] 6.2474ms 5.9465ms 168.1654 Ops/s 168.0495 Ops/s $\color{#35bf28}+0.07\%$
test_ppo_speed[False-backward] 12.8921ms 12.6012ms 79.3573 Ops/s 81.6708 Ops/s $\color{#d91a1a}-2.83\%$
test_ppo_speed[True-None] 4.0330ms 3.6330ms 275.2573 Ops/s 273.4956 Ops/s $\color{#35bf28}+0.64\%$
test_ppo_speed[True-backward] 8.8531ms 8.4772ms 117.9631 Ops/s 117.7551 Ops/s $\color{#35bf28}+0.18\%$
test_ppo_speed[reduce-overhead-None] 3.7893ms 3.6523ms 273.8001 Ops/s 274.5741 Ops/s $\color{#d91a1a}-0.28\%$
test_ppo_speed[reduce-overhead-backward] 8.9514ms 8.6550ms 115.5406 Ops/s 114.4598 Ops/s $\color{#35bf28}+0.94\%$
test_reinforce_speed[False-None] 4.9594ms 4.6389ms 215.5700 Ops/s 220.7621 Ops/s $\color{#d91a1a}-2.35\%$
test_reinforce_speed[False-backward] 7.9572ms 7.4431ms 134.3528 Ops/s 134.6402 Ops/s $\color{#d91a1a}-0.21\%$
test_reinforce_speed[True-None] 3.0968ms 2.9118ms 343.4286 Ops/s 342.5357 Ops/s $\color{#35bf28}+0.26\%$
test_reinforce_speed[True-backward] 8.0602ms 7.6985ms 129.8953 Ops/s 131.9230 Ops/s $\color{#d91a1a}-1.54\%$
test_reinforce_speed[reduce-overhead-None] 3.2581ms 2.8686ms 348.6055 Ops/s 351.6390 Ops/s $\color{#d91a1a}-0.86\%$
test_reinforce_speed[reduce-overhead-backward] 8.3350ms 7.8973ms 126.6261 Ops/s 121.0887 Ops/s $\color{#35bf28}+4.57\%$
test_iql_speed[False-None] 20.8736ms 19.9220ms 50.1958 Ops/s 49.8473 Ops/s $\color{#35bf28}+0.70\%$
test_iql_speed[False-backward] 36.1976ms 30.6330ms 32.6445 Ops/s 32.8843 Ops/s $\color{#d91a1a}-0.73\%$
test_iql_speed[True-None] 8.7350ms 8.4627ms 118.1654 Ops/s 115.4522 Ops/s $\color{#35bf28}+2.35\%$
test_iql_speed[True-backward] 17.1100ms 16.7405ms 59.7354 Ops/s 56.7795 Ops/s $\textbf{\color{#35bf28}+5.21\%}$
test_iql_speed[reduce-overhead-None] 8.7696ms 8.5267ms 117.2784 Ops/s 115.0808 Ops/s $\color{#35bf28}+1.91\%$
test_iql_speed[reduce-overhead-backward] 17.4705ms 17.0786ms 58.5529 Ops/s 57.7062 Ops/s $\color{#35bf28}+1.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4710ms 6.1147ms 163.5394 Ops/s 167.0280 Ops/s $\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6279ms 0.3543ms 2.8221 KOps/s 2.6195 KOps/s $\textbf{\color{#35bf28}+7.73\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5301ms 0.3229ms 3.0974 KOps/s 2.7196 KOps/s $\textbf{\color{#35bf28}+13.89\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0822ms 5.7754ms 173.1470 Ops/s 174.9017 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7188ms 0.3378ms 2.9602 KOps/s 2.8473 KOps/s $\color{#35bf28}+3.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5595ms 0.2991ms 3.3437 KOps/s 2.9976 KOps/s $\textbf{\color{#35bf28}+11.55\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6377ms 1.4116ms 708.4253 Ops/s 811.9140 Ops/s $\textbf{\color{#d91a1a}-12.75\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5820ms 1.3249ms 754.7956 Ops/s 866.3167 Ops/s $\textbf{\color{#d91a1a}-12.87\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.9377ms 6.1350ms 162.9992 Ops/s 170.2474 Ops/s $\color{#d91a1a}-4.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1563ms 0.4753ms 2.1041 KOps/s 2.0302 KOps/s $\color{#35bf28}+3.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7162ms 0.4572ms 2.1872 KOps/s 2.0871 KOps/s $\color{#35bf28}+4.79\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0741ms 5.8006ms 172.3950 Ops/s 175.3526 Ops/s $\color{#d91a1a}-1.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.1179ms 0.3195ms 3.1296 KOps/s 767.6732 Ops/s $\textbf{\color{#35bf28}+307.67\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5852ms 0.3273ms 3.0556 KOps/s 2.7218 KOps/s $\textbf{\color{#35bf28}+12.27\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9365ms 5.7320ms 174.4603 Ops/s 174.1583 Ops/s $\color{#35bf28}+0.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6845ms 0.3445ms 2.9028 KOps/s 2.6486 KOps/s $\textbf{\color{#35bf28}+9.60\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5619ms 0.3260ms 3.0674 KOps/s 2.7505 KOps/s $\textbf{\color{#35bf28}+11.52\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0238ms 5.9138ms 169.0968 Ops/s 168.4141 Ops/s $\color{#35bf28}+0.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8543ms 0.5064ms 1.9749 KOps/s 1.8557 KOps/s $\textbf{\color{#35bf28}+6.42\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7355ms 0.4847ms 2.0632 KOps/s 2.1256 KOps/s $\color{#d91a1a}-2.93\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6907ms 5.2604ms 190.0994 Ops/s 194.1174 Ops/s $\color{#d91a1a}-2.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.5456ms 2.2691ms 440.7003 Ops/s 441.6334 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2340ms 1.2098ms 826.5797 Ops/s 902.0909 Ops/s $\textbf{\color{#d91a1a}-8.37\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5099s 15.3356ms 65.2079 Ops/s 55.2656 Ops/s $\textbf{\color{#35bf28}+17.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.1380ms 2.0534ms 486.9913 Ops/s 579.8905 Ops/s $\textbf{\color{#d91a1a}-16.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.4382ms 1.0787ms 927.0646 Ops/s 829.3998 Ops/s $\textbf{\color{#35bf28}+11.78\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 9.5862ms 5.4410ms 183.7892 Ops/s 185.8230 Ops/s $\color{#d91a1a}-1.09\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.6255ms 2.1616ms 462.6203 Ops/s 461.7166 Ops/s $\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 9.2131ms 1.3732ms 728.2132 Ops/s 730.9939 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.2039ms 33.1905ms 30.1291 Ops/s 30.0502 Ops/s $\color{#35bf28}+0.26\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.5839ms 17.5536ms 56.9684 Ops/s 56.5339 Ops/s $\color{#35bf28}+0.77\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 37.3753ms 34.4295ms 29.0449 Ops/s 29.0345 Ops/s $\color{#35bf28}+0.04\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.9847ms 17.5349ms 57.0290 Ops/s 55.7716 Ops/s $\color{#35bf28}+2.25\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 37.7325ms 35.9526ms 27.8144 Ops/s 27.8078 Ops/s $\color{#35bf28}+0.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.1260ms 18.9426ms 52.7912 Ops/s 52.0984 Ops/s $\color{#35bf28}+1.33\%$

@vmoens
Copy link
Collaborator Author

vmoens commented Oct 26, 2025

Merged in #3220

@vmoens vmoens closed this Oct 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. llm/refactor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens