Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix hold_out_net #1719

Merged
merged 1 commit into from
Nov 28, 2023
Merged

[BugFix] Fix hold_out_net #1719

merged 1 commit into from
Nov 28, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 28, 2023 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 28, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1719

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (5 Unrelated Failures)

As of commit 5277b05 with merge base 07fcfb1 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 28, 2023
@github-actions GitHub Actions
Copy link

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}38$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.6715ms 62.8990ms 15.8985 Ops/s 14.7796 Ops/s $\textbf{\color{#35bf28}+7.57\%}$
test_sync 39.1521ms 34.7917ms 28.7425 Ops/s 27.2660 Ops/s $\textbf{\color{#35bf28}+5.42\%}$
test_async 57.1990ms 33.0391ms 30.2672 Ops/s 29.1278 Ops/s $\color{#35bf28}+3.91\%$
test_simple 0.4970s 0.4420s 2.2624 Ops/s 2.2092 Ops/s $\color{#35bf28}+2.41\%$
test_transformed 0.6452s 0.5953s 1.6798 Ops/s 1.6101 Ops/s $\color{#35bf28}+4.33\%$
test_serial 1.3597s 1.3223s 0.7562 Ops/s 0.7336 Ops/s $\color{#35bf28}+3.09\%$
test_parallel 1.3940s 1.3540s 0.7385 Ops/s 0.7443 Ops/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[True-True-True-True-True] 0.1564ms 22.7700μs 43.9174 KOps/s 43.4893 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-True-True-True-False] 39.2730μs 13.8129μs 72.3963 KOps/s 72.1683 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-True-False-True] 0.1615ms 14.0446μs 71.2019 KOps/s 71.1246 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-True-False-False] 30.4060μs 8.3815μs 119.3100 KOps/s 117.1152 KOps/s $\color{#35bf28}+1.87\%$
test_step_mdp_speed[True-True-False-True-True] 0.2823ms 24.1220μs 41.4559 KOps/s 40.9557 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-True-False-True-False] 95.8180μs 15.0619μs 66.3927 KOps/s 65.8020 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-True-False-False-True] 47.8090μs 15.0007μs 66.6637 KOps/s 65.3145 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-True-False-False-False] 31.3680μs 9.6868μs 103.2335 KOps/s 102.3240 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-True-True-True] 64.6400μs 25.4270μs 39.3283 KOps/s 38.5106 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-True-True-False] 46.0160μs 16.6324μs 60.1236 KOps/s 59.9768 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-True-False-True] 46.5260μs 15.2852μs 65.4226 KOps/s 65.2322 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[True-False-True-False-False] 45.1240μs 9.8507μs 101.5157 KOps/s 101.9242 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-True-True] 62.4160μs 26.4765μs 37.7693 KOps/s 37.2301 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-False-False-True-False] 46.0660μs 17.7227μs 56.4248 KOps/s 55.9602 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-False-False-True] 40.7160μs 16.3454μs 61.1792 KOps/s 60.6060 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-False-False-False-False] 65.9100μs 11.0182μs 90.7586 KOps/s 90.1874 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-True-True-True] 58.3390μs 25.6448μs 38.9943 KOps/s 38.4369 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-False] 63.3070μs 16.6069μs 60.2158 KOps/s 59.4243 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[False-True-True-False-True] 47.1680μs 17.2697μs 57.9047 KOps/s 56.3232 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[False-True-True-False-False] 43.1400μs 10.8695μs 92.0008 KOps/s 88.7027 KOps/s $\color{#35bf28}+3.72\%$
test_step_mdp_speed[False-True-False-True-True] 53.5090μs 26.5022μs 37.7327 KOps/s 36.7131 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-False-True-False] 42.5700μs 17.7760μs 56.2558 KOps/s 55.8633 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-True-False-False-True] 55.1930μs 18.4062μs 54.3296 KOps/s 53.0420 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-True-False-False-False] 33.9740μs 12.2211μs 81.8258 KOps/s 82.2713 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-True-True-True] 58.3280μs 27.9288μs 35.8054 KOps/s 34.8214 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-False-True-True-False] 47.4170μs 19.0853μs 52.3962 KOps/s 51.9380 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-False-True] 52.9490μs 18.6720μs 53.5560 KOps/s 52.4416 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[False-False-True-False-False] 1.6198ms 12.2387μs 81.7082 KOps/s 81.2926 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-False-True-True] 58.7090μs 28.6700μs 34.8796 KOps/s 33.9905 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[False-False-False-True-False] 51.4050μs 20.2625μs 49.3523 KOps/s 49.3282 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-False-False-True] 47.4280μs 19.3570μs 51.6609 KOps/s 50.0874 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[False-False-False-False-False] 35.8570μs 13.0379μs 76.6994 KOps/s 75.1500 KOps/s $\color{#35bf28}+2.06\%$
test_values[generalized_advantage_estimate-True-True] 13.6661ms 12.0996ms 82.6476 Ops/s 75.3050 Ops/s $\textbf{\color{#35bf28}+9.75\%}$
test_values[vec_generalized_advantage_estimate-True-True] 48.5979ms 26.8644ms 37.2240 Ops/s 36.6323 Ops/s $\color{#35bf28}+1.62\%$
test_values[td0_return_estimate-False-False] 0.2268ms 0.1763ms 5.6713 KOps/s 5.0749 KOps/s $\textbf{\color{#35bf28}+11.75\%}$
test_values[td1_return_estimate-False-False] 28.9859ms 26.1942ms 38.1764 Ops/s 37.6806 Ops/s $\color{#35bf28}+1.32\%$
test_values[vec_td1_return_estimate-False-False] 34.0405ms 26.5214ms 37.7054 Ops/s 37.0078 Ops/s $\color{#35bf28}+1.88\%$
test_values[td_lambda_return_estimate-True-False] 37.1783ms 36.5741ms 27.3418 Ops/s 26.7332 Ops/s $\color{#35bf28}+2.28\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.1162ms 26.8668ms 37.2207 Ops/s 36.5264 Ops/s $\color{#35bf28}+1.90\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2637ms 8.0526ms 124.1836 Ops/s 119.2311 Ops/s $\color{#35bf28}+4.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2511ms 1.8363ms 544.5856 Ops/s 509.3637 Ops/s $\textbf{\color{#35bf28}+6.91\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.3377ms 0.4327ms 2.3109 KOps/s 2.2615 KOps/s $\color{#35bf28}+2.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.8273ms 39.5493ms 25.2849 Ops/s 24.9388 Ops/s $\color{#35bf28}+1.39\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 10.6880ms 2.5001ms 399.9864 Ops/s 394.7171 Ops/s $\color{#35bf28}+1.33\%$
test_dqn_speed 9.3661ms 1.6215ms 616.7213 Ops/s 567.8443 Ops/s $\textbf{\color{#35bf28}+8.61\%}$
test_ddpg_speed 11.4198ms 3.6104ms 276.9811 Ops/s 265.1816 Ops/s $\color{#35bf28}+4.45\%$
test_sac_speed 18.8321ms 10.2215ms 97.8333 Ops/s 89.3777 Ops/s $\textbf{\color{#35bf28}+9.46\%}$
test_redq_speed 25.3480ms 19.1950ms 52.0970 Ops/s 49.1970 Ops/s $\textbf{\color{#35bf28}+5.89\%}$
test_redq_deprec_speed 88.6259ms 16.2886ms 61.3925 Ops/s 59.8661 Ops/s $\color{#35bf28}+2.55\%$
test_td3_speed 18.3069ms 10.4214ms 95.9567 Ops/s 87.2150 Ops/s $\textbf{\color{#35bf28}+10.02\%}$
test_cql_speed 46.1024ms 38.7498ms 25.8066 Ops/s 24.6492 Ops/s $\color{#35bf28}+4.70\%$
test_a2c_speed 16.9432ms 8.0839ms 123.7027 Ops/s 110.2529 Ops/s $\textbf{\color{#35bf28}+12.20\%}$
test_ppo_speed 16.9244ms 8.4372ms 118.5222 Ops/s 107.4793 Ops/s $\textbf{\color{#35bf28}+10.27\%}$
test_reinforce_speed 15.4009ms 7.1860ms 139.1603 Ops/s 125.5579 Ops/s $\textbf{\color{#35bf28}+10.83\%}$
test_iql_speed 42.2329ms 34.0095ms 29.4036 Ops/s 27.9632 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.1209ms 1.8639ms 536.5138 Ops/s 455.2262 Ops/s $\textbf{\color{#35bf28}+17.86\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.2236ms 1.9750ms 506.3372 Ops/s 432.1777 Ops/s $\textbf{\color{#35bf28}+17.16\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.1281ms 2.0842ms 479.7914 Ops/s 432.4745 Ops/s $\textbf{\color{#35bf28}+10.94\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.8436ms 1.9307ms 517.9550 Ops/s 466.2305 Ops/s $\textbf{\color{#35bf28}+11.09\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.9773ms 1.9901ms 502.4930 Ops/s 430.5077 Ops/s $\textbf{\color{#35bf28}+16.72\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.5014ms 1.9923ms 501.9440 Ops/s 431.4831 Ops/s $\textbf{\color{#35bf28}+16.33\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.4268ms 1.8919ms 528.5637 Ops/s 464.2506 Ops/s $\textbf{\color{#35bf28}+13.85\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.0352ms 1.9905ms 502.3973 Ops/s 400.4039 Ops/s $\textbf{\color{#35bf28}+25.47\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0135ms 2.0338ms 491.6841 Ops/s 424.8948 Ops/s $\textbf{\color{#35bf28}+15.72\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7278ms 1.8707ms 534.5656 Ops/s 464.8622 Ops/s $\textbf{\color{#35bf28}+14.99\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.3248ms 2.1238ms 470.8553 Ops/s 438.0837 Ops/s $\textbf{\color{#35bf28}+7.48\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2.9010ms 1.9927ms 501.8435 Ops/s 469.5147 Ops/s $\textbf{\color{#35bf28}+6.89\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4622ms 1.8728ms 533.9642 Ops/s 516.5813 Ops/s $\color{#35bf28}+3.36\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.7976ms 1.9988ms 500.3077 Ops/s 477.2958 Ops/s $\color{#35bf28}+4.82\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2.5299ms 2.0128ms 496.8264 Ops/s 458.2240 Ops/s $\textbf{\color{#35bf28}+8.42\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5098ms 1.8528ms 539.7278 Ops/s 466.6043 Ops/s $\textbf{\color{#35bf28}+15.67\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.1628ms 2.0138ms 496.5783 Ops/s 434.9662 Ops/s $\textbf{\color{#35bf28}+14.16\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.0897ms 1.9712ms 507.3117 Ops/s 453.0746 Ops/s $\textbf{\color{#35bf28}+11.97\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1708s 17.5362ms 57.0249 Ops/s 50.9844 Ops/s $\textbf{\color{#35bf28}+11.85\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1225s 16.4629ms 60.7425 Ops/s 56.0441 Ops/s $\textbf{\color{#35bf28}+8.38\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1030s 15.9625ms 62.6467 Ops/s 54.8179 Ops/s $\textbf{\color{#35bf28}+14.28\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1038s 15.9887ms 62.5443 Ops/s 54.3662 Ops/s $\textbf{\color{#35bf28}+15.04\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1021s 16.0016ms 62.4939 Ops/s 54.3630 Ops/s $\textbf{\color{#35bf28}+14.96\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1186s 16.4095ms 60.9402 Ops/s 48.8991 Ops/s $\textbf{\color{#35bf28}+24.62\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1054s 16.2329ms 61.6034 Ops/s 54.4110 Ops/s $\textbf{\color{#35bf28}+13.22\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1131s 16.2656ms 61.4793 Ops/s 55.6564 Ops/s $\textbf{\color{#35bf28}+10.46\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1064s 16.1545ms 61.9022 Ops/s 55.8370 Ops/s $\textbf{\color{#35bf28}+10.86\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1206s 0.1202s 8.3179 Ops/s 7.9543 Ops/s $\color{#35bf28}+4.57\%$
test_sync 0.1049s 0.1030s 9.7108 Ops/s 9.7260 Ops/s $\color{#d91a1a}-0.16\%$
test_async 0.2690s 0.1005s 9.9517 Ops/s 9.9021 Ops/s $\color{#35bf28}+0.50\%$
test_single_pixels 0.1437s 0.1433s 6.9775 Ops/s 6.9105 Ops/s $\color{#35bf28}+0.97\%$
test_sync_pixels 98.2667ms 95.5233ms 10.4687 Ops/s 10.5316 Ops/s $\color{#d91a1a}-0.60\%$
test_async_pixels 0.2463s 91.9310ms 10.8777 Ops/s 10.8870 Ops/s $\color{#d91a1a}-0.08\%$
test_simple 0.9352s 0.8696s 1.1499 Ops/s 1.1354 Ops/s $\color{#35bf28}+1.29\%$
test_transformed 1.1771s 1.1156s 0.8964 Ops/s 0.8870 Ops/s $\color{#35bf28}+1.06\%$
test_serial 2.4939s 2.4430s 0.4093 Ops/s 0.4083 Ops/s $\color{#35bf28}+0.25\%$
test_parallel 2.5713s 2.4867s 0.4021 Ops/s 0.3983 Ops/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-True-True-True-True] 0.1021ms 34.0662μs 29.3546 KOps/s 28.6555 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[True-True-True-True-False] 42.3800μs 19.8181μs 50.4589 KOps/s 47.7248 KOps/s $\textbf{\color{#35bf28}+5.73\%}$
test_step_mdp_speed[True-True-True-False-True] 42.4910μs 19.7229μs 50.7024 KOps/s 49.1082 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[True-True-True-False-False] 26.6800μs 11.9219μs 83.8794 KOps/s 82.4117 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-True-False-True-True] 67.6010μs 35.9944μs 27.7821 KOps/s 27.1625 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-False-True-False] 46.9400μs 21.7074μs 46.0671 KOps/s 44.4498 KOps/s $\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-True-False-False-True] 55.1610μs 21.9925μs 45.4700 KOps/s 45.1458 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-False] 32.8200μs 13.6621μs 73.1952 KOps/s 70.8024 KOps/s $\color{#35bf28}+3.38\%$
test_step_mdp_speed[True-False-True-True-True] 62.8300μs 38.0592μs 26.2749 KOps/s 25.5856 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[True-False-True-True-False] 45.9400μs 23.4730μs 42.6022 KOps/s 42.1107 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-False-True] 43.2610μs 22.1135μs 45.2212 KOps/s 45.1953 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-False-True-False-False] 38.7500μs 13.5584μs 73.7551 KOps/s 71.3787 KOps/s $\color{#35bf28}+3.33\%$
test_step_mdp_speed[True-False-False-True-True] 70.4700μs 39.6940μs 25.1928 KOps/s 24.5900 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[True-False-False-True-False] 48.7910μs 25.4217μs 39.3365 KOps/s 38.3247 KOps/s $\color{#35bf28}+2.64\%$
test_step_mdp_speed[True-False-False-False-True] 43.1510μs 24.3401μs 41.0844 KOps/s 40.9329 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-False-False-False] 38.1400μs 15.3056μs 65.3355 KOps/s 61.7167 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_step_mdp_speed[False-True-True-True-True] 62.4710μs 37.0925μs 26.9596 KOps/s 25.5339 KOps/s $\textbf{\color{#35bf28}+5.58\%}$
test_step_mdp_speed[False-True-True-True-False] 42.8500μs 23.6607μs 42.2641 KOps/s 41.3171 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-True-False-True] 52.0300μs 26.1444μs 38.2491 KOps/s 37.7717 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-True-False-False] 36.6700μs 15.5149μs 64.4540 KOps/s 63.2594 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-True-False-True-True] 67.3910μs 40.1403μs 24.9126 KOps/s 24.6116 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-True-False] 53.6510μs 25.4137μs 39.3489 KOps/s 38.4927 KOps/s $\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-True-False-False-True] 53.7710μs 27.8214μs 35.9435 KOps/s 36.2267 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-True-False-False-False] 63.2510μs 17.4534μs 57.2953 KOps/s 56.5978 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-True-True-True] 65.0400μs 41.7258μs 23.9660 KOps/s 22.9196 KOps/s $\color{#35bf28}+4.57\%$
test_step_mdp_speed[False-False-True-True-False] 52.8110μs 27.2513μs 36.6955 KOps/s 35.8671 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-False-True-False-True] 64.0900μs 28.1572μs 35.5149 KOps/s 36.1501 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[False-False-True-False-False] 38.6210μs 17.3638μs 57.5911 KOps/s 56.8824 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-True] 77.9210μs 43.5402μs 22.9673 KOps/s 22.8305 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-False-True-False] 55.0100μs 29.2170μs 34.2267 KOps/s 33.5362 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-False-False-False-True] 80.1410μs 29.7017μs 33.6682 KOps/s 34.0349 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-False-False-False] 38.1900μs 19.1343μs 52.2623 KOps/s 51.8511 KOps/s $\color{#35bf28}+0.79\%$
test_values[generalized_advantage_estimate-True-True] 26.7438ms 25.9058ms 38.6014 Ops/s 38.4227 Ops/s $\color{#35bf28}+0.46\%$
test_values[vec_generalized_advantage_estimate-True-True] 87.3655ms 3.3111ms 302.0115 Ops/s 292.4080 Ops/s $\color{#35bf28}+3.28\%$
test_values[td0_return_estimate-False-False] 99.8210μs 65.4611μs 15.2762 KOps/s 14.9053 KOps/s $\color{#35bf28}+2.49\%$
test_values[td1_return_estimate-False-False] 58.0451ms 57.4366ms 17.4105 Ops/s 17.2458 Ops/s $\color{#35bf28}+0.96\%$
test_values[vec_td1_return_estimate-False-False] 2.0371ms 1.7333ms 576.9477 Ops/s 565.3974 Ops/s $\color{#35bf28}+2.04\%$
test_values[td_lambda_return_estimate-True-False] 91.6951ms 90.7572ms 11.0184 Ops/s 10.8326 Ops/s $\color{#35bf28}+1.72\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0297ms 1.7393ms 574.9429 Ops/s 571.1725 Ops/s $\color{#35bf28}+0.66\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.5629ms 26.3739ms 37.9163 Ops/s 37.8278 Ops/s $\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8807ms 0.7267ms 1.3761 KOps/s 1.3658 KOps/s $\color{#35bf28}+0.75\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7338ms 0.6818ms 1.4667 KOps/s 1.4492 KOps/s $\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5297ms 1.4788ms 676.2338 Ops/s 672.6086 Ops/s $\color{#35bf28}+0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9778ms 0.7122ms 1.4041 KOps/s 1.3932 KOps/s $\color{#35bf28}+0.78\%$
test_dqn_speed 7.9590ms 1.4684ms 681.0111 Ops/s 669.5175 Ops/s $\color{#35bf28}+1.72\%$
test_ddpg_speed 6.1186ms 3.3408ms 299.3278 Ops/s 297.8699 Ops/s $\color{#35bf28}+0.49\%$
test_sac_speed 10.0431ms 9.3080ms 107.4346 Ops/s 98.3091 Ops/s $\textbf{\color{#35bf28}+9.28\%}$
test_redq_speed 16.9625ms 16.5959ms 60.2557 Ops/s 60.0064 Ops/s $\color{#35bf28}+0.42\%$
test_redq_deprec_speed 14.6246ms 13.1313ms 76.1538 Ops/s 75.9185 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed 18.3240ms 9.5257ms 104.9790 Ops/s 104.5350 Ops/s $\color{#35bf28}+0.42\%$
test_cql_speed 33.6705ms 32.3970ms 30.8671 Ops/s 31.0205 Ops/s $\color{#d91a1a}-0.49\%$
test_a2c_speed 0.1062s 7.9760ms 125.3766 Ops/s 135.9512 Ops/s $\textbf{\color{#d91a1a}-7.78\%}$
test_ppo_speed 8.9902ms 7.5305ms 132.7939 Ops/s 130.9210 Ops/s $\color{#35bf28}+1.43\%$
test_reinforce_speed 7.8837ms 6.2807ms 159.2180 Ops/s 159.0967 Ops/s $\color{#35bf28}+0.08\%$
test_iql_speed 28.8807ms 27.4132ms 36.4788 Ops/s 36.7462 Ops/s $\color{#d91a1a}-0.73\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0860ms 2.4784ms 403.4829 Ops/s 396.6510 Ops/s $\color{#35bf28}+1.72\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.7670ms 2.6555ms 376.5751 Ops/s 367.9371 Ops/s $\color{#35bf28}+2.35\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.0305ms 2.6359ms 379.3806 Ops/s 367.8756 Ops/s $\color{#35bf28}+3.13\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0514ms 2.4427ms 409.3775 Ops/s 399.9672 Ops/s $\color{#35bf28}+2.35\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.5412ms 2.6517ms 377.1126 Ops/s 367.9844 Ops/s $\color{#35bf28}+2.48\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.0394ms 2.6504ms 377.3002 Ops/s 369.7711 Ops/s $\color{#35bf28}+2.04\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1540ms 2.4429ms 409.3462 Ops/s 397.2771 Ops/s $\color{#35bf28}+3.04\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.7709ms 2.6501ms 377.3403 Ops/s 369.2400 Ops/s $\color{#35bf28}+2.19\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0298ms 2.6526ms 376.9896 Ops/s 366.9394 Ops/s $\color{#35bf28}+2.74\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1694ms 2.4689ms 405.0423 Ops/s 395.7832 Ops/s $\color{#35bf28}+2.34\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.5401ms 2.6363ms 379.3158 Ops/s 367.8216 Ops/s $\color{#35bf28}+3.12\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5702ms 2.6494ms 377.4495 Ops/s 367.8294 Ops/s $\color{#35bf28}+2.62\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6498ms 2.4340ms 410.8393 Ops/s 396.9177 Ops/s $\color{#35bf28}+3.51\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.8462ms 2.6638ms 375.4095 Ops/s 368.3216 Ops/s $\color{#35bf28}+1.92\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8580ms 2.6472ms 377.7538 Ops/s 368.3727 Ops/s $\color{#35bf28}+2.55\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8791ms 2.4492ms 408.2895 Ops/s 396.9183 Ops/s $\color{#35bf28}+2.86\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.1265ms 2.6537ms 376.8281 Ops/s 367.1591 Ops/s $\color{#35bf28}+2.63\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.6591ms 2.6534ms 376.8683 Ops/s 368.8242 Ops/s $\color{#35bf28}+2.18\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2189s 19.3291ms 51.7355 Ops/s 51.5905 Ops/s $\color{#35bf28}+0.28\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1248s 15.1179ms 66.1469 Ops/s 65.1245 Ops/s $\color{#35bf28}+1.57\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1247s 15.1078ms 66.1912 Ops/s 56.9547 Ops/s $\textbf{\color{#35bf28}+16.22\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1267s 17.3477ms 57.6444 Ops/s 56.8746 Ops/s $\color{#35bf28}+1.35\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1248s 17.4170ms 57.4150 Ops/s 56.8272 Ops/s $\color{#35bf28}+1.03\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1249s 17.4212ms 57.4012 Ops/s 56.9247 Ops/s $\color{#35bf28}+0.84\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1271s 17.3682ms 57.5767 Ops/s 56.9798 Ops/s $\color{#35bf28}+1.05\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1265s 17.3995ms 57.4728 Ops/s 65.3705 Ops/s $\textbf{\color{#d91a1a}-12.08\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1253s 15.1711ms 65.9149 Ops/s 56.7638 Ops/s $\textbf{\color{#35bf28}+16.12\%}$

@vmoens vmoens marked this pull request as ready for review November 28, 2023 16:46
@vmoens vmoens merged commit 2a72e6d into main Nov 28, 2023
56 of 61 checks passed
@vmoens vmoens deleted the fix-param-detach branch November 28, 2023 16:46
@vmoens vmoens added the bug Something isn't working label Nov 28, 2023
@albertbou92 albertbou92 mentioned this pull request Dec 5, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@albertbou92 albertbou92 albertbou92 approved these changes

Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @albertbou92 @facebook-github-bot