Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revert "[BugFix] buffer __iter__ for samplers without replacement + prefetch" #2182

Merged
merged 1 commit into from
May 30, 2024
Merged

Revert "[BugFix] buffer __iter__ for samplers without replacement + prefetch" #2182

merged 1 commit into from
May 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented May 30, 2024 •
edited
Loading

Reverts #2178

@JulianKu I'm reverting this because it breaks the offline IQL example

To test this:

python sota-implementations/iql/iql_offline.py \
  optim.gradient_steps=55 \
  logger.backend=

Feel free to reopen and we'll find a solution!

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented May 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2182

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 2 New Failures, 15 Unrelated Failures

As of commit f584cd3 with merge base 0f02c4a (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 30, 2024
@vmoens vmoens merged commit d59b810 into main May 30, 2024
14 of 23 checks passed
@vmoens vmoens deleted the revert-2178-fix-iter-prefetch branch May 30, 2024 16:10
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1217s 59.9033ms 16.6936 Ops/s 17.7543 Ops/s $\textbf{\color{#d91a1a}-5.97\%}$
test_sync 39.3570ms 36.5576ms 27.3541 Ops/s 31.7273 Ops/s $\textbf{\color{#d91a1a}-13.78\%}$
test_async 58.3480ms 29.1656ms 34.2870 Ops/s 32.2641 Ops/s $\textbf{\color{#35bf28}+6.27\%}$
test_simple 0.4504s 0.3871s 2.5833 Ops/s 2.7387 Ops/s $\textbf{\color{#d91a1a}-5.67\%}$
test_transformed 0.6020s 0.5456s 1.8330 Ops/s 1.8967 Ops/s $\color{#d91a1a}-3.36\%$
test_serial 1.3130s 1.2519s 0.7988 Ops/s 0.7795 Ops/s $\color{#35bf28}+2.48\%$
test_parallel 1.1414s 1.0612s 0.9423 Ops/s 0.9370 Ops/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-True-True-True-True] 0.1502ms 21.9805μs 45.4949 KOps/s 46.0941 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-True-True-False] 40.2960μs 13.4336μs 74.4403 KOps/s 76.0371 KOps/s $\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-True-True-False-True] 44.0630μs 12.9815μs 77.0324 KOps/s 77.8246 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-True-False-False] 36.2680μs 7.8553μs 127.3034 KOps/s 130.8075 KOps/s $\color{#d91a1a}-2.68\%$
test_step_mdp_speed[True-True-False-True-True] 88.2650μs 23.4431μs 42.6564 KOps/s 42.7640 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-True-False-True-False] 38.8230μs 14.6981μs 68.0359 KOps/s 69.0705 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-True-False-False-True] 54.9120μs 14.2999μs 69.9306 KOps/s 70.2890 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[True-True-False-False-False] 47.7500μs 9.1570μs 109.2059 KOps/s 110.9793 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[True-False-True-True-True] 52.6380μs 25.0413μs 39.9341 KOps/s 40.2867 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-False-True-True-False] 54.5220μs 16.1074μs 62.0834 KOps/s 63.5786 KOps/s $\color{#d91a1a}-2.35\%$
test_step_mdp_speed[True-False-True-False-True] 40.4760μs 14.4016μs 69.4365 KOps/s 70.0539 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-False-True-False-False] 37.5110μs 9.2757μs 107.8082 KOps/s 110.5373 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-False-False-True-True] 68.5590μs 26.1584μs 38.2286 KOps/s 38.7492 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-False-False-True-False] 40.9870μs 17.2258μs 58.0523 KOps/s 59.1749 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-False-False-False-True] 49.5030μs 15.4875μs 64.5683 KOps/s 65.4325 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-False-False-False-False] 47.7590μs 10.4444μs 95.7449 KOps/s 98.7978 KOps/s $\color{#d91a1a}-3.09\%$
test_step_mdp_speed[False-True-True-True-True] 51.7870μs 24.7347μs 40.4290 KOps/s 40.5681 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[False-True-True-True-False] 43.9720μs 16.1604μs 61.8796 KOps/s 63.6549 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-True-True-False-True] 39.8750μs 16.6931μs 59.9051 KOps/s 60.7671 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[False-True-True-False-False] 34.8460μs 10.5361μs 94.9114 KOps/s 97.8567 KOps/s $\color{#d91a1a}-3.01\%$
test_step_mdp_speed[False-True-False-True-True] 83.9770μs 25.9098μs 38.5955 KOps/s 38.7683 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-False-True-False] 50.1040μs 17.2438μs 57.9918 KOps/s 59.2651 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-True-False-False-True] 37.4100μs 17.9590μs 55.6825 KOps/s 57.0541 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-False-False-False] 38.1120μs 11.6486μs 85.8475 KOps/s 87.7339 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-False-True-True-True] 66.6040μs 27.0794μs 36.9284 KOps/s 36.9789 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-False-True-True-False] 53.8200μs 18.6344μs 53.6643 KOps/s 54.7478 KOps/s $\color{#d91a1a}-1.98\%$
test_step_mdp_speed[False-False-True-False-True] 68.2370μs 17.8188μs 56.1206 KOps/s 56.9948 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[False-False-True-False-False] 36.6790μs 11.7750μs 84.9255 KOps/s 87.4983 KOps/s $\color{#d91a1a}-2.94\%$
test_step_mdp_speed[False-False-False-True-True] 48.0600μs 29.0278μs 34.4497 KOps/s 34.9845 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[False-False-False-True-False] 55.9440μs 19.8353μs 50.4152 KOps/s 51.8619 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-False-False-False-True] 39.5240μs 19.0116μs 52.5994 KOps/s 53.8034 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-False-False-False-False] 55.1140μs 12.8627μs 77.7439 KOps/s 80.2849 KOps/s $\color{#d91a1a}-3.16\%$
test_values[generalized_advantage_estimate-True-True] 9.9329ms 9.4924ms 105.3477 Ops/s 99.6380 Ops/s $\textbf{\color{#35bf28}+5.73\%}$
test_values[vec_generalized_advantage_estimate-True-True] 38.9838ms 35.7242ms 27.9922 Ops/s 29.6531 Ops/s $\textbf{\color{#d91a1a}-5.60\%}$
test_values[td0_return_estimate-False-False] 0.2351ms 0.1691ms 5.9133 KOps/s 4.8761 KOps/s $\textbf{\color{#35bf28}+21.27\%}$
test_values[td1_return_estimate-False-False] 26.3711ms 23.6013ms 42.3705 Ops/s 40.4922 Ops/s $\color{#35bf28}+4.64\%$
test_values[vec_td1_return_estimate-False-False] 42.1614ms 35.6444ms 28.0549 Ops/s 29.2708 Ops/s $\color{#d91a1a}-4.15\%$
test_values[td_lambda_return_estimate-True-False] 36.5601ms 33.9951ms 29.4160 Ops/s 28.4905 Ops/s $\color{#35bf28}+3.25\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.5498ms 35.4199ms 28.2327 Ops/s 29.6226 Ops/s $\color{#d91a1a}-4.69\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5914ms 8.3586ms 119.6368 Ops/s 117.3023 Ops/s $\color{#35bf28}+1.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.0684ms 1.8654ms 536.0879 Ops/s 542.7785 Ops/s $\color{#d91a1a}-1.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4720ms 0.3575ms 2.7969 KOps/s 2.6934 KOps/s $\color{#35bf28}+3.84\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 51.1257ms 45.4409ms 22.0066 Ops/s 21.8323 Ops/s $\color{#35bf28}+0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6253ms 3.0384ms 329.1257 Ops/s 319.4323 Ops/s $\color{#35bf28}+3.03\%$
test_dqn_speed 1.8194ms 1.3700ms 729.9130 Ops/s 703.7471 Ops/s $\color{#35bf28}+3.72\%$
test_ddpg_speed 3.9770ms 2.9097ms 343.6788 Ops/s 332.3550 Ops/s $\color{#35bf28}+3.41\%$
test_sac_speed 8.9822ms 8.5813ms 116.5329 Ops/s 107.5540 Ops/s $\textbf{\color{#35bf28}+8.35\%}$
test_redq_speed 14.7045ms 13.3414ms 74.9548 Ops/s 70.2090 Ops/s $\textbf{\color{#35bf28}+6.76\%}$
test_redq_deprec_speed 15.9141ms 14.1841ms 70.5016 Ops/s 64.3964 Ops/s $\textbf{\color{#35bf28}+9.48\%}$
test_td3_speed 10.1721ms 8.9821ms 111.3325 Ops/s 107.6582 Ops/s $\color{#35bf28}+3.41\%$
test_cql_speed 39.6804ms 38.0523ms 26.2796 Ops/s 26.2378 Ops/s $\color{#35bf28}+0.16\%$
test_a2c_speed 9.5809ms 7.9929ms 125.1106 Ops/s 124.2624 Ops/s $\color{#35bf28}+0.68\%$
test_ppo_speed 9.2961ms 8.1419ms 122.8212 Ops/s 118.2735 Ops/s $\color{#35bf28}+3.85\%$
test_reinforce_speed 7.8920ms 6.9174ms 144.5637 Ops/s 140.2736 Ops/s $\color{#35bf28}+3.06\%$
test_iql_speed 35.2647ms 33.7670ms 29.6147 Ops/s 28.9854 Ops/s $\color{#35bf28}+2.17\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.5710ms 3.8914ms 256.9756 Ops/s 260.1630 Ops/s $\color{#d91a1a}-1.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8986ms 0.5059ms 1.9768 KOps/s 1.9559 KOps/s $\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5434ms 0.4887ms 2.0462 KOps/s 2.0786 KOps/s $\color{#d91a1a}-1.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.6135ms 3.9508ms 253.1116 Ops/s 263.8635 Ops/s $\color{#d91a1a}-4.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8066ms 0.5035ms 1.9861 KOps/s 1.9494 KOps/s $\color{#35bf28}+1.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.5979ms 0.4721ms 2.1181 KOps/s 2.0708 KOps/s $\color{#35bf28}+2.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7107ms 1.2471ms 801.8799 Ops/s 760.9951 Ops/s $\textbf{\color{#35bf28}+5.37\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 0.1150s 1.3559ms 737.5370 Ops/s 711.2702 Ops/s $\color{#35bf28}+3.69\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.8082ms 3.9093ms 255.7998 Ops/s 248.3246 Ops/s $\color{#35bf28}+3.01\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9640ms 0.6185ms 1.6168 KOps/s 1.5629 KOps/s $\color{#35bf28}+3.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.7751ms 0.6118ms 1.6344 KOps/s 1.6513 KOps/s $\color{#d91a1a}-1.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.5123ms 3.9051ms 256.0758 Ops/s 258.6262 Ops/s $\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6091ms 0.5088ms 1.9655 KOps/s 1.5064 KOps/s $\textbf{\color{#35bf28}+30.48\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6314ms 0.4711ms 2.1226 KOps/s 2.0176 KOps/s $\textbf{\color{#35bf28}+5.20\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.1144ms 3.6963ms 270.5405 Ops/s 249.2238 Ops/s $\textbf{\color{#35bf28}+8.55\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9588ms 0.4932ms 2.0278 KOps/s 1.9241 KOps/s $\textbf{\color{#35bf28}+5.39\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8914ms 0.4695ms 2.1297 KOps/s 2.0325 KOps/s $\color{#35bf28}+4.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5412ms 3.8080ms 262.6032 Ops/s 239.9813 Ops/s $\textbf{\color{#35bf28}+9.43\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0569ms 0.6224ms 1.6066 KOps/s 1.5431 KOps/s $\color{#35bf28}+4.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7085ms 0.5851ms 1.7091 KOps/s 1.6098 KOps/s $\textbf{\color{#35bf28}+6.17\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1221s 6.2131ms 160.9504 Ops/s 162.2383 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 20.8342ms 12.6094ms 79.3062 Ops/s 75.0039 Ops/s $\textbf{\color{#35bf28}+5.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.7738ms 1.1022ms 907.2536 Ops/s 907.7925 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1143s 8.1255ms 123.0698 Ops/s 122.9715 Ops/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.9686ms 12.5015ms 79.9904 Ops/s 74.7631 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7724ms 1.1000ms 909.0664 Ops/s 891.3806 Ops/s $\color{#35bf28}+1.98\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 96.9693ms 5.9353ms 168.4832 Ops/s 157.2391 Ops/s $\textbf{\color{#35bf28}+7.15\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.9503ms 12.8996ms 77.5219 Ops/s 75.3929 Ops/s $\color{#35bf28}+2.82\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0985ms 1.4320ms 698.3228 Ops/s 670.4885 Ops/s $\color{#35bf28}+4.15\%$

@github-actions GitHub Actions
Copy link

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1194s 0.1174s 8.5143 Ops/s 8.6880 Ops/s $\color{#d91a1a}-2.00\%$
test_sync 0.1005s 99.4762ms 10.0527 Ops/s 9.8629 Ops/s $\color{#35bf28}+1.92\%$
test_async 0.1991s 99.3179ms 10.0687 Ops/s 10.2969 Ops/s $\color{#d91a1a}-2.22\%$
test_single_pixels 0.1269s 0.1266s 7.8962 Ops/s 7.8703 Ops/s $\color{#35bf28}+0.33\%$
test_sync_pixels 85.8146ms 81.8508ms 12.2173 Ops/s 12.2446 Ops/s $\color{#d91a1a}-0.22\%$
test_async_pixels 0.1585s 69.2435ms 14.4418 Ops/s 15.0715 Ops/s $\color{#d91a1a}-4.18\%$
test_simple 0.7716s 0.7707s 1.2975 Ops/s 1.2904 Ops/s $\color{#35bf28}+0.55\%$
test_transformed 1.0390s 1.0324s 0.9686 Ops/s 0.9631 Ops/s $\color{#35bf28}+0.57\%$
test_serial 2.5026s 2.4415s 0.4096 Ops/s 0.4079 Ops/s $\color{#35bf28}+0.41\%$
test_parallel 2.3931s 2.3244s 0.4302 Ops/s 0.4253 Ops/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-True-True-True] 0.1306ms 33.7598μs 29.6210 KOps/s 29.0045 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-True-False] 0.2119ms 20.3110μs 49.2345 KOps/s 48.8556 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-True-True-False-True] 0.1504ms 19.3818μs 51.5949 KOps/s 49.7148 KOps/s $\color{#35bf28}+3.78\%$
test_step_mdp_speed[True-True-True-False-False] 0.2217ms 11.5614μs 86.4944 KOps/s 84.6442 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[True-True-False-True-True] 0.2447ms 35.5524μs 28.1275 KOps/s 27.8139 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-True-False-True-False] 0.2377ms 22.0245μs 45.4040 KOps/s 44.9922 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-False-True] 0.1407ms 21.2436μs 47.0731 KOps/s 45.5329 KOps/s $\color{#35bf28}+3.38\%$
test_step_mdp_speed[True-True-False-False-False] 0.2109ms 13.5288μs 73.9164 KOps/s 72.8680 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-False-True-True-True] 0.2428ms 37.7022μs 26.5236 KOps/s 26.0429 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-False-True-True-False] 64.1410μs 24.1707μs 41.3723 KOps/s 40.9417 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-False-True-False-True] 53.5920μs 21.1224μs 47.3430 KOps/s 45.6161 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[True-False-True-False-False] 34.7900μs 13.5817μs 73.6286 KOps/s 72.2196 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-False-False-True-True] 0.2457ms 38.9662μs 25.6632 KOps/s 24.9423 KOps/s $\color{#35bf28}+2.89\%$
test_step_mdp_speed[True-False-False-True-False] 0.2125ms 25.8491μs 38.6860 KOps/s 38.0811 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-False-False-False-True] 0.1006ms 23.0569μs 43.3710 KOps/s 42.3609 KOps/s $\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-False-False-False-False] 0.2072ms 15.3526μs 65.1354 KOps/s 64.3348 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[False-True-True-True-True] 0.2247ms 37.5754μs 26.6131 KOps/s 25.9095 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[False-True-True-True-False] 50.4720μs 24.1551μs 41.3992 KOps/s 41.0107 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-True-True-False-True] 0.2279ms 25.2627μs 39.5841 KOps/s 38.3599 KOps/s $\color{#35bf28}+3.19\%$
test_step_mdp_speed[False-True-True-False-False] 0.2159ms 15.6933μs 63.7215 KOps/s 63.4778 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[False-True-False-True-True] 0.2273ms 39.3403μs 25.4192 KOps/s 24.9323 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-False-True-False] 46.8000μs 25.9282μs 38.5681 KOps/s 38.3407 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[False-True-False-False-True] 0.2275ms 27.2604μs 36.6833 KOps/s 36.3934 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-True-False-False-False] 0.2061ms 17.5615μs 56.9426 KOps/s 57.5378 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[False-False-True-True-True] 0.3532ms 41.8140μs 23.9154 KOps/s 23.5396 KOps/s $\color{#35bf28}+1.60\%$
test_step_mdp_speed[False-False-True-True-False] 0.2255ms 28.2642μs 35.3804 KOps/s 35.1751 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-True-False-True] 63.2110μs 27.2863μs 36.6485 KOps/s 36.9872 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-False-True-False-False] 0.2204ms 17.4667μs 57.2519 KOps/s 58.3722 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-False-False-True-True] 63.8810μs 43.9331μs 22.7619 KOps/s 22.4166 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[False-False-False-True-False] 0.2335ms 29.6286μs 33.7512 KOps/s 33.1643 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-False-False-False-True] 0.2154ms 28.6479μs 34.9066 KOps/s 33.6836 KOps/s $\color{#35bf28}+3.63\%$
test_step_mdp_speed[False-False-False-False-False] 48.5110μs 19.1732μs 52.1561 KOps/s 51.6650 KOps/s $\color{#35bf28}+0.95\%$
test_values[generalized_advantage_estimate-True-True] 25.7177ms 24.0551ms 41.5713 Ops/s 41.6020 Ops/s $\color{#d91a1a}-0.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 90.5707ms 3.3695ms 296.7787 Ops/s 307.2320 Ops/s $\color{#d91a1a}-3.40\%$
test_values[td0_return_estimate-False-False] 93.8710μs 63.0186μs 15.8683 KOps/s 16.3242 KOps/s $\color{#d91a1a}-2.79\%$
test_values[td1_return_estimate-False-False] 53.7268ms 51.3418ms 19.4773 Ops/s 19.0940 Ops/s $\color{#35bf28}+2.01\%$
test_values[vec_td1_return_estimate-False-False] 2.1220ms 1.7468ms 572.4790 Ops/s 573.0227 Ops/s $\color{#d91a1a}-0.09\%$
test_values[td_lambda_return_estimate-True-False] 88.0177ms 82.3151ms 12.1484 Ops/s 11.9500 Ops/s $\color{#35bf28}+1.66\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0958ms 1.7417ms 574.1617 Ops/s 570.6642 Ops/s $\color{#35bf28}+0.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.7673ms 22.5363ms 44.3728 Ops/s 43.1412 Ops/s $\color{#35bf28}+2.85\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8774ms 0.6796ms 1.4715 KOps/s 1.4726 KOps/s $\color{#d91a1a}-0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7836ms 0.6342ms 1.5768 KOps/s 1.5739 KOps/s $\color{#35bf28}+0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6041ms 1.4418ms 693.5901 Ops/s 695.6954 Ops/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9354ms 0.6494ms 1.5398 KOps/s 1.5268 KOps/s $\color{#35bf28}+0.85\%$
test_dqn_speed 1.8376ms 1.4431ms 692.9680 Ops/s 636.1402 Ops/s $\textbf{\color{#35bf28}+8.93\%}$
test_ddpg_speed 3.1954ms 2.9644ms 337.3398 Ops/s 334.6002 Ops/s $\color{#35bf28}+0.82\%$
test_sac_speed 8.9506ms 8.5288ms 117.2499 Ops/s 116.2696 Ops/s $\color{#35bf28}+0.84\%$
test_redq_speed 12.0841ms 10.7710ms 92.8420 Ops/s 91.0764 Ops/s $\color{#35bf28}+1.94\%$
test_redq_deprec_speed 12.7268ms 12.1364ms 82.3970 Ops/s 84.3477 Ops/s $\color{#d91a1a}-2.31\%$
test_td3_speed 17.8110ms 8.5268ms 117.2776 Ops/s 116.2807 Ops/s $\color{#35bf28}+0.86\%$
test_cql_speed 28.5737ms 26.6865ms 37.4721 Ops/s 38.0989 Ops/s $\color{#d91a1a}-1.65\%$
test_a2c_speed 6.1034ms 5.8507ms 170.9183 Ops/s 172.1594 Ops/s $\color{#d91a1a}-0.72\%$
test_ppo_speed 6.5033ms 6.1709ms 162.0503 Ops/s 164.4282 Ops/s $\color{#d91a1a}-1.45\%$
test_reinforce_speed 5.1372ms 4.8654ms 205.5341 Ops/s 210.8692 Ops/s $\color{#d91a1a}-2.53\%$
test_iql_speed 21.1017ms 20.4170ms 48.9789 Ops/s 49.2933 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.8625ms 4.6625ms 214.4754 Ops/s 211.8932 Ops/s $\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8466ms 0.6020ms 1.6611 KOps/s 1.6580 KOps/s $\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.0540ms 0.5804ms 1.7229 KOps/s 1.7072 KOps/s $\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0143ms 4.6381ms 215.6040 Ops/s 213.3863 Ops/s $\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8363ms 0.5935ms 1.6849 KOps/s 1.6706 KOps/s $\color{#35bf28}+0.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.8917ms 0.5760ms 1.7362 KOps/s 1.7273 KOps/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0123ms 1.5417ms 648.6417 Ops/s 641.6670 Ops/s $\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.7914ms 1.4794ms 675.9527 Ops/s 665.6192 Ops/s $\color{#35bf28}+1.55\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9451ms 4.7822ms 209.1102 Ops/s 208.1229 Ops/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6108ms 0.7383ms 1.3545 KOps/s 1.3526 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9507ms 0.7161ms 1.3964 KOps/s 1.4155 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9725ms 4.7008ms 212.7286 Ops/s 211.3074 Ops/s $\color{#35bf28}+0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8360ms 0.6105ms 1.6380 KOps/s 1.6604 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.2593ms 0.5961ms 1.6776 KOps/s 1.7166 KOps/s $\color{#d91a1a}-2.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0710ms 4.6768ms 213.8227 Ops/s 215.2165 Ops/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7987ms 0.6075ms 1.6462 KOps/s 1.6933 KOps/s $\color{#d91a1a}-2.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7859ms 0.5897ms 1.6958 KOps/s 1.7608 KOps/s $\color{#d91a1a}-3.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.0116ms 4.8400ms 206.6123 Ops/s 207.8780 Ops/s $\color{#d91a1a}-0.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9088ms 0.7319ms 1.3663 KOps/s 1.3604 KOps/s $\color{#35bf28}+0.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.3931ms 0.7181ms 1.3925 KOps/s 1.3971 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1366s 7.5646ms 132.1953 Ops/s 136.6565 Ops/s $\color{#d91a1a}-3.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.6458ms 16.4921ms 60.6352 Ops/s 60.9021 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3917ms 1.2449ms 803.2570 Ops/s 809.6029 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1234s 9.6758ms 103.3507 Ops/s 102.4517 Ops/s $\color{#35bf28}+0.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.2536ms 16.3571ms 61.1355 Ops/s 60.7784 Ops/s $\color{#35bf28}+0.59\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.3715ms 1.2545ms 797.1044 Ops/s 814.5971 Ops/s $\color{#d91a1a}-2.15\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1261s 7.7264ms 129.4258 Ops/s 129.4251 Ops/s $+0.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.8250ms 16.2025ms 61.7189 Ops/s 59.4085 Ops/s $\color{#35bf28}+3.89\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.9381ms 1.6460ms 607.5261 Ops/s 610.6071 Ops/s $\color{#d91a1a}-0.50\%$

@JulianKu JulianKu mentioned this pull request May 31, 2024
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vmoens @facebook-github-bot