Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Pin transformers version to < 4.42.0 to make vmap happy #2278

Merged
merged 1 commit into from
Jul 9, 2024
Merged

[CI] Pin transformers version to < 4.42.0 to make vmap happy #2278

merged 1 commit into from
Jul 9, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 9, 2024

With transformers>=4.42.0 we get this error from calling vmap around the model:

  File "/home/vmoens/tensordict/tensordict/nn/common.py", line 1185, in forward
    tensors = self._call_module(tensors, **kwargs)
  File "/home/vmoens/tensordict/tensordict/nn/common.py", line 1141, in _call_module
    out = self.module(*tensors, **kwargs)
  File "/home/vmoens/.conda/envs/torchrl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1566, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/vmoens/.conda/envs/torchrl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1575, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/vmoens/.conda/envs/torchrl/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1144, in forward
    attention_mask = _prepare_4d_causal_attention_mask_for_sdpa(
  File "/home/vmoens/.conda/envs/torchrl/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py", line 372, in _prepare_4d_causal_attention_mask_for_sdpa
    ignore_causal_mask = AttentionMaskConverter._ignore_causal_mask_sdpa(
  File "/home/vmoens/.conda/envs/torchrl/lib/python3.10/site-packages/transformers/modeling_attn_mask_utils.py", line 279, in _ignore_causal_mask_sdpa
    elif (is_training or not is_tracing) and torch.all(attention_mask == 1):
RuntimeError: vmap: It looks like you're attempting to use a Tensor in some data-dependent control flow. We don't support that yet, please shout over at https://github.com/pytorch/functorch/issues/257 .

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 9, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2278

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 6 Unrelated Failures

As of commit 1f425b5 with merge base a151923 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 9, 2024
@vmoens vmoens changed the title init [CI] Pin transformers version to < 4.42.0 to make vmap happy Jul 9, 2024
@vmoens vmoens added the bug Something isn't working label Jul 9, 2024
@vmoens vmoens merged commit 533d3f9 into main Jul 9, 2024
41 of 51 checks passed
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jul 9, 2024
@vmoens vmoens deleted the fix-rlhf2 branch July 9, 2024 10:00
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1239s 60.7141ms 16.4706 Ops/s 17.6823 Ops/s $\textbf{\color{#d91a1a}-6.85\%}$
test_sync 48.9632ms 33.3001ms 30.0299 Ops/s 31.2029 Ops/s $\color{#d91a1a}-3.76\%$
test_async 52.5957ms 29.2536ms 34.1839 Ops/s 34.2011 Ops/s $\color{#d91a1a}-0.05\%$
test_simple 0.3881s 0.3860s 2.5909 Ops/s 2.5915 Ops/s $\color{#d91a1a}-0.02\%$
test_transformed 0.5440s 0.5414s 1.8469 Ops/s 1.8101 Ops/s $\color{#35bf28}+2.04\%$
test_serial 1.3549s 1.2915s 0.7743 Ops/s 0.7891 Ops/s $\color{#d91a1a}-1.87\%$
test_parallel 1.1649s 1.1082s 0.9024 Ops/s 0.9144 Ops/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[True-True-True-True-True] 0.2292ms 22.6450μs 44.1599 KOps/s 44.2140 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-True-True-False] 53.0100μs 13.1180μs 76.2313 KOps/s 75.5816 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-True-False-True] 39.7550μs 13.0539μs 76.6055 KOps/s 75.7580 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-True-True-False-False] 33.3320μs 7.6820μs 130.1744 KOps/s 128.5879 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-False-True-True] 68.0880μs 24.0331μs 41.6093 KOps/s 41.4370 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-True-False-True-False] 40.7470μs 14.3373μs 69.7480 KOps/s 68.1942 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-False-False-True] 45.4450μs 14.3786μs 69.5476 KOps/s 69.4829 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[True-True-False-False-False] 31.4190μs 8.9386μs 111.8748 KOps/s 110.1432 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-True-True-True] 62.1370μs 25.7306μs 38.8642 KOps/s 38.9987 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-False-True-True-False] 39.3240μs 15.7216μs 63.6068 KOps/s 62.9041 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-False-True-False-True] 40.9770μs 14.4427μs 69.2393 KOps/s 69.5991 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-True-False-False] 56.8070μs 8.9504μs 111.7264 KOps/s 110.1168 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-False-False-True-True] 58.8410μs 26.5963μs 37.5992 KOps/s 37.7086 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-False-True-False] 56.8670μs 16.7213μs 59.8038 KOps/s 58.3264 KOps/s $\color{#35bf28}+2.53\%$
test_step_mdp_speed[True-False-False-False-True] 39.0840μs 15.6201μs 64.0202 KOps/s 64.2240 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-False-False-False] 42.6710μs 10.0897μs 99.1110 KOps/s 96.9307 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[False-True-True-True-True] 88.8940μs 25.2206μs 39.6502 KOps/s 39.1675 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-True-True-True-False] 57.9190μs 15.7337μs 63.5579 KOps/s 62.4905 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-True-True-False-True] 51.7680μs 16.7564μs 59.6788 KOps/s 59.8109 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-True-True-False-False] 36.0370μs 10.1424μs 98.5964 KOps/s 96.5605 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-True-False-True-True] 87.0740μs 26.6714μs 37.4934 KOps/s 37.4521 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[False-True-False-True-False] 45.4860μs 16.8629μs 59.3016 KOps/s 58.2915 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-True-False-False-True] 66.3850μs 17.9811μs 55.6141 KOps/s 56.0620 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-True-False-False-False] 34.9560μs 11.4966μs 86.9823 KOps/s 88.6821 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-False-True-True-True] 68.5890μs 27.9162μs 35.8215 KOps/s 36.0796 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[False-False-True-True-False] 47.6100μs 18.1033μs 55.2386 KOps/s 54.2723 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-False-True-False-True] 70.8430μs 17.8202μs 56.1161 KOps/s 56.1525 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-False-True-False-False] 53.0300μs 11.3985μs 87.7305 KOps/s 87.0064 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-False-False-True-True] 47.3790μs 29.3267μs 34.0986 KOps/s 34.1533 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-False-False-True-False] 53.6010μs 19.1368μs 52.2554 KOps/s 51.4083 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-False-False-False-True] 57.2780μs 19.0216μs 52.5719 KOps/s 53.3225 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[False-False-False-False-False] 41.3380μs 12.4010μs 80.6386 KOps/s 79.4808 KOps/s $\color{#35bf28}+1.46\%$
test_values[generalized_advantage_estimate-True-True] 11.1625ms 9.7440ms 102.6271 Ops/s 104.5817 Ops/s $\color{#d91a1a}-1.87\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.1845ms 33.7648ms 29.6166 Ops/s 28.2016 Ops/s $\textbf{\color{#35bf28}+5.02\%}$
test_values[td0_return_estimate-False-False] 0.2240ms 0.1820ms 5.4942 KOps/s 5.6252 KOps/s $\color{#d91a1a}-2.33\%$
test_values[td1_return_estimate-False-False] 24.7198ms 24.0672ms 41.5503 Ops/s 41.8992 Ops/s $\color{#d91a1a}-0.83\%$
test_values[vec_td1_return_estimate-False-False] 36.0854ms 34.1523ms 29.2806 Ops/s 28.1062 Ops/s $\color{#35bf28}+4.18\%$
test_values[td_lambda_return_estimate-True-False] 37.9321ms 35.1629ms 28.4390 Ops/s 28.7297 Ops/s $\color{#d91a1a}-1.01\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.0919ms 33.9383ms 29.4652 Ops/s 28.1421 Ops/s $\color{#35bf28}+4.70\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5841ms 8.3472ms 119.8004 Ops/s 121.6489 Ops/s $\color{#d91a1a}-1.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4807ms 2.0250ms 493.8340 Ops/s 439.5050 Ops/s $\textbf{\color{#35bf28}+12.36\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6011ms 0.3700ms 2.7024 KOps/s 2.8233 KOps/s $\color{#d91a1a}-4.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 55.1522ms 44.3918ms 22.5267 Ops/s 21.5855 Ops/s $\color{#35bf28}+4.36\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6222ms 3.0659ms 326.1690 Ops/s 326.4337 Ops/s $\color{#d91a1a}-0.08\%$
test_dqn_speed 2.0726ms 1.3762ms 726.6511 Ops/s 745.9525 Ops/s $\color{#d91a1a}-2.59\%$
test_ddpg_speed 3.7036ms 2.9411ms 340.0127 Ops/s 350.9251 Ops/s $\color{#d91a1a}-3.11\%$
test_sac_speed 9.5571ms 8.9129ms 112.1974 Ops/s 116.6866 Ops/s $\color{#d91a1a}-3.85\%$
test_redq_speed 15.7147ms 14.0610ms 71.1187 Ops/s 67.2238 Ops/s $\textbf{\color{#35bf28}+5.79\%}$
test_redq_deprec_speed 16.0477ms 14.3627ms 69.6246 Ops/s 72.2333 Ops/s $\color{#d91a1a}-3.61\%$
test_td3_speed 18.8884ms 8.9029ms 112.3234 Ops/s 117.8531 Ops/s $\color{#d91a1a}-4.69\%$
test_cql_speed 38.2193ms 37.3655ms 26.7626 Ops/s 27.5444 Ops/s $\color{#d91a1a}-2.84\%$
test_a2c_speed 9.4381ms 7.6620ms 130.5141 Ops/s 131.8576 Ops/s $\color{#d91a1a}-1.02\%$
test_ppo_speed 10.6410ms 8.1635ms 122.4966 Ops/s 127.1744 Ops/s $\color{#d91a1a}-3.68\%$
test_reinforce_speed 8.4051ms 6.7700ms 147.7114 Ops/s 149.5365 Ops/s $\color{#d91a1a}-1.22\%$
test_iql_speed 34.3941ms 33.0902ms 30.2204 Ops/s 30.5705 Ops/s $\color{#d91a1a}-1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9372ms 3.6987ms 270.3680 Ops/s 275.2756 Ops/s $\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2514ms 0.5071ms 1.9720 KOps/s 1.9718 KOps/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7369ms 0.4794ms 2.0862 KOps/s 2.0742 KOps/s $\color{#35bf28}+0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.5028ms 3.7230ms 268.6010 Ops/s 279.5298 Ops/s $\color{#d91a1a}-3.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0723ms 0.4988ms 2.0049 KOps/s 2.0149 KOps/s $\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8779ms 0.4712ms 2.1222 KOps/s 2.0909 KOps/s $\color{#35bf28}+1.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4497ms 1.7478ms 572.1320 Ops/s 578.6152 Ops/s $\color{#d91a1a}-1.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.7079ms 1.6652ms 600.5437 Ops/s 609.3524 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.6685ms 3.8494ms 259.7804 Ops/s 270.7155 Ops/s $\color{#d91a1a}-4.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1230s 0.7551ms 1.3244 KOps/s 1.5644 KOps/s $\textbf{\color{#d91a1a}-15.34\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7819ms 0.6161ms 1.6232 KOps/s 1.6458 KOps/s $\color{#d91a1a}-1.37\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.4271ms 3.6874ms 271.1926 Ops/s 276.4557 Ops/s $\color{#d91a1a}-1.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4548ms 0.5028ms 1.9887 KOps/s 1.9381 KOps/s $\color{#35bf28}+2.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8374ms 0.4828ms 2.0712 KOps/s 2.0784 KOps/s $\color{#d91a1a}-0.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8330ms 3.6707ms 272.4254 Ops/s 272.2898 Ops/s $\color{#35bf28}+0.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7898ms 0.5027ms 1.9892 KOps/s 1.9722 KOps/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8492ms 0.4816ms 2.0765 KOps/s 2.0534 KOps/s $\color{#35bf28}+1.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0337ms 3.8208ms 261.7276 Ops/s 268.1279 Ops/s $\color{#d91a1a}-2.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7700ms 0.6442ms 1.5523 KOps/s 1.5598 KOps/s $\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8569ms 0.6169ms 1.6210 KOps/s 1.5891 KOps/s $\color{#35bf28}+2.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1305s 6.3825ms 156.6772 Ops/s 121.0528 Ops/s $\textbf{\color{#35bf28}+29.43\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.4582ms 12.5997ms 79.3672 Ops/s 80.3594 Ops/s $\color{#d91a1a}-1.23\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.1626ms 1.0727ms 932.2305 Ops/s 949.0618 Ops/s $\color{#d91a1a}-1.77\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1287s 8.3496ms 119.7659 Ops/s 163.3003 Ops/s $\textbf{\color{#d91a1a}-26.66\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.3243ms 12.5330ms 79.7894 Ops/s 79.8421 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.5433ms 1.1256ms 888.4334 Ops/s 947.0900 Ops/s $\textbf{\color{#d91a1a}-6.19\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1128s 6.1414ms 162.8306 Ops/s 164.1095 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.4253ms 12.8296ms 77.9445 Ops/s 66.4995 Ops/s $\textbf{\color{#35bf28}+17.21\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7121ms 1.2223ms 818.1257 Ops/s 836.7194 Ops/s $\color{#d91a1a}-2.22\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

$\color{#D29922}\textsf{\Large&amp;#x26A0;\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1611s 0.1081s 9.2541 Ops/s 9.8909 Ops/s $\textbf{\color{#d91a1a}-6.44\%}$
test_sync 92.8105ms 89.0020ms 11.2357 Ops/s 11.2363 Ops/s $-0.01\%$
test_async 0.1716s 86.3890ms 11.5755 Ops/s 13.6615 Ops/s $\textbf{\color{#d91a1a}-15.27\%}$
test_single_pixels 0.1155s 0.1139s 8.7833 Ops/s 8.8994 Ops/s $\color{#d91a1a}-1.30\%$
test_sync_pixels 69.7096ms 67.6492ms 14.7821 Ops/s 14.6873 Ops/s $\color{#35bf28}+0.65\%$
test_async_pixels 0.1250s 55.9885ms 17.8608 Ops/s 17.8773 Ops/s $\color{#d91a1a}-0.09\%$
test_simple 0.7272s 0.7268s 1.3759 Ops/s 1.3671 Ops/s $\color{#35bf28}+0.65\%$
test_transformed 1.0313s 0.9739s 1.0268 Ops/s 1.0415 Ops/s $\color{#d91a1a}-1.41\%$
test_serial 2.2103s 2.1497s 0.4652 Ops/s 0.4713 Ops/s $\color{#d91a1a}-1.29\%$
test_parallel 1.8767s 1.8437s 0.5424 Ops/s 0.5408 Ops/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-True-True-True-True] 87.7920μs 33.9337μs 29.4692 KOps/s 30.3952 KOps/s $\color{#d91a1a}-3.05\%$
test_step_mdp_speed[True-True-True-True-False] 35.2810μs 19.6835μs 50.8039 KOps/s 52.0351 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[True-True-True-False-True] 43.3610μs 19.4981μs 51.2870 KOps/s 53.9547 KOps/s $\color{#d91a1a}-4.94\%$
test_step_mdp_speed[True-True-True-False-False] 36.7910μs 11.0964μs 90.1192 KOps/s 91.9780 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-True-False-True-True] 57.6610μs 35.0805μs 28.5058 KOps/s 29.2694 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-True-False-True-False] 39.5610μs 21.4890μs 46.5353 KOps/s 47.9580 KOps/s $\color{#d91a1a}-2.97\%$
test_step_mdp_speed[True-True-False-False-True] 39.5910μs 20.6231μs 48.4893 KOps/s 49.7642 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[True-True-False-False-False] 28.1310μs 12.8752μs 77.6689 KOps/s 78.7262 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-False-True-True-True] 56.5710μs 37.2137μs 26.8718 KOps/s 27.9436 KOps/s $\color{#d91a1a}-3.84\%$
test_step_mdp_speed[True-False-True-True-False] 41.9310μs 23.4887μs 42.5737 KOps/s 44.1233 KOps/s $\color{#d91a1a}-3.51\%$
test_step_mdp_speed[True-False-True-False-True] 37.0010μs 20.9113μs 47.8211 KOps/s 49.5990 KOps/s $\color{#d91a1a}-3.58\%$
test_step_mdp_speed[True-False-True-False-False] 31.1600μs 12.9065μs 77.4802 KOps/s 78.4188 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[True-False-False-True-True] 60.3510μs 38.8050μs 25.7699 KOps/s 26.1358 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-False-False-True-False] 43.0310μs 25.0059μs 39.9906 KOps/s 39.6766 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[True-False-False-False-True] 49.0410μs 22.7377μs 43.9799 KOps/s 45.7416 KOps/s $\color{#d91a1a}-3.85\%$
test_step_mdp_speed[True-False-False-False-False] 32.1910μs 14.6231μs 68.3850 KOps/s 69.2009 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[False-True-True-True-True] 56.0910μs 37.4360μs 26.7122 KOps/s 27.7063 KOps/s $\color{#d91a1a}-3.59\%$
test_step_mdp_speed[False-True-True-True-False] 44.0410μs 23.5836μs 42.4024 KOps/s 43.9161 KOps/s $\color{#d91a1a}-3.45\%$
test_step_mdp_speed[False-True-True-False-True] 43.0910μs 24.9727μs 40.0438 KOps/s 41.1373 KOps/s $\color{#d91a1a}-2.66\%$
test_step_mdp_speed[False-True-True-False-False] 38.8210μs 14.5452μs 68.7510 KOps/s 69.2997 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-False-True-True] 78.8920μs 38.2583μs 26.1381 KOps/s 26.5339 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[False-True-False-True-False] 41.7410μs 24.8085μs 40.3087 KOps/s 40.6180 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-True-False-False-True] 53.3610μs 26.4901μs 37.7500 KOps/s 38.8842 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[False-True-False-False-False] 38.8410μs 16.4481μs 60.7972 KOps/s 61.7656 KOps/s $\color{#d91a1a}-1.57\%$
test_step_mdp_speed[False-False-True-True-True] 57.4610μs 41.1964μs 24.2739 KOps/s 25.2198 KOps/s $\color{#d91a1a}-3.75\%$
test_step_mdp_speed[False-False-True-True-False] 50.9810μs 27.0203μs 37.0092 KOps/s 37.5156 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[False-False-True-False-True] 48.1810μs 26.9536μs 37.1008 KOps/s 38.8692 KOps/s $\color{#d91a1a}-4.55\%$
test_step_mdp_speed[False-False-True-False-False] 38.0510μs 16.5708μs 60.3472 KOps/s 61.4559 KOps/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[False-False-False-True-True] 65.1210μs 43.2179μs 23.1386 KOps/s 23.6187 KOps/s $\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-False-False-True-False] 52.7310μs 29.1564μs 34.2977 KOps/s 34.4284 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-False-False-False-True] 56.4110μs 27.8950μs 35.8487 KOps/s 36.3803 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[False-False-False-False-False] 35.5610μs 18.4387μs 54.2337 KOps/s 55.9396 KOps/s $\color{#d91a1a}-3.05\%$
test_values[generalized_advantage_estimate-True-True] 25.5102ms 23.8849ms 41.8675 Ops/s 43.4128 Ops/s $\color{#d91a1a}-3.56\%$
test_values[vec_generalized_advantage_estimate-True-True] 95.5845ms 2.8022ms 356.8566 Ops/s 351.5534 Ops/s $\color{#35bf28}+1.51\%$
test_values[td0_return_estimate-False-False] 90.2620μs 65.3817μs 15.2948 KOps/s 15.8603 KOps/s $\color{#d91a1a}-3.57\%$
test_values[td1_return_estimate-False-False] 57.7504ms 54.7407ms 18.2680 Ops/s 19.1538 Ops/s $\color{#d91a1a}-4.63\%$
test_values[vec_td1_return_estimate-False-False] 1.3572ms 1.0759ms 929.4167 Ops/s 945.4685 Ops/s $\color{#d91a1a}-1.70\%$
test_values[td_lambda_return_estimate-True-False] 86.1873ms 85.9381ms 11.6363 Ops/s 12.0896 Ops/s $\color{#d91a1a}-3.75\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3766ms 1.0722ms 932.6662 Ops/s 946.9342 Ops/s $\color{#d91a1a}-1.51\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.4845ms 23.3490ms 42.8284 Ops/s 42.1223 Ops/s $\color{#35bf28}+1.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9400ms 0.7111ms 1.4062 KOps/s 1.3915 KOps/s $\color{#35bf28}+1.06\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7776ms 0.6594ms 1.5165 KOps/s 1.5507 KOps/s $\color{#d91a1a}-2.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4834ms 1.4618ms 684.1023 Ops/s 692.0424 Ops/s $\color{#d91a1a}-1.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.6966ms 0.6724ms 1.4873 KOps/s 1.5172 KOps/s $\color{#d91a1a}-1.97\%$
test_dqn_speed 7.7318ms 1.4699ms 680.2957 Ops/s 692.1994 Ops/s $\color{#d91a1a}-1.72\%$
test_ddpg_speed 3.2594ms 3.0013ms 333.1882 Ops/s 346.6235 Ops/s $\color{#d91a1a}-3.88\%$
test_sac_speed 9.2541ms 8.5824ms 116.5174 Ops/s 119.7835 Ops/s $\color{#d91a1a}-2.73\%$
test_redq_speed 0.1042s 11.8605ms 84.3134 Ops/s 94.1532 Ops/s $\textbf{\color{#d91a1a}-10.45\%}$
test_redq_deprec_speed 12.0407ms 11.5602ms 86.5040 Ops/s 78.6546 Ops/s $\textbf{\color{#35bf28}+9.98\%}$
test_td3_speed 8.6975ms 8.4857ms 117.8457 Ops/s 120.3281 Ops/s $\color{#d91a1a}-2.06\%$
test_cql_speed 26.5342ms 25.7938ms 38.7690 Ops/s 39.1286 Ops/s $\color{#d91a1a}-0.92\%$
test_a2c_speed 5.9760ms 5.6939ms 175.6270 Ops/s 174.2335 Ops/s $\color{#35bf28}+0.80\%$
test_ppo_speed 6.2440ms 5.9882ms 166.9957 Ops/s 165.2573 Ops/s $\color{#35bf28}+1.05\%$
test_reinforce_speed 5.5740ms 4.6700ms 214.1311 Ops/s 212.4714 Ops/s $\color{#35bf28}+0.78\%$
test_iql_speed 20.3481ms 19.6782ms 50.8176 Ops/s 50.8202 Ops/s $-0.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7520ms 4.6079ms 217.0204 Ops/s 220.2793 Ops/s $\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1081s 0.6304ms 1.5863 KOps/s 1.6202 KOps/s $\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7291ms 0.5230ms 1.9119 KOps/s 1.9347 KOps/s $\color{#d91a1a}-1.18\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.8265ms 4.5974ms 217.5121 Ops/s 221.4817 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4530ms 0.5398ms 1.8524 KOps/s 1.8710 KOps/s $\color{#d91a1a}-1.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6681ms 0.5158ms 1.9387 KOps/s 1.9552 KOps/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1670ms 1.9886ms 502.8618 Ops/s 504.9293 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0709ms 1.8742ms 533.5618 Ops/s 544.8769 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8846ms 4.7504ms 210.5102 Ops/s 212.7055 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3718ms 0.6936ms 1.4418 KOps/s 1.4545 KOps/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8413ms 0.6699ms 1.4927 KOps/s 1.5031 KOps/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7924ms 4.6042ms 217.1952 Ops/s 218.8729 Ops/s $\color{#d91a1a}-0.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6882ms 0.5421ms 1.8447 KOps/s 1.8545 KOps/s $\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5591ms 0.5262ms 1.9005 KOps/s 1.9056 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7658ms 4.5905ms 217.8414 Ops/s 221.1819 Ops/s $\color{#d91a1a}-1.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6561ms 0.5411ms 1.8482 KOps/s 1.8660 KOps/s $\color{#d91a1a}-0.95\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6365ms 0.5171ms 1.9338 KOps/s 1.9277 KOps/s $\color{#35bf28}+0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8452ms 4.7640ms 209.9063 Ops/s 212.0265 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3357ms 0.6996ms 1.4294 KOps/s 1.4525 KOps/s $\color{#d91a1a}-1.59\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8738ms 0.6741ms 1.4834 KOps/s 1.5150 KOps/s $\color{#d91a1a}-2.09\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1463s 7.6495ms 130.7283 Ops/s 132.0298 Ops/s $\color{#d91a1a}-0.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.8916ms 15.6357ms 63.9563 Ops/s 66.6573 Ops/s $\color{#d91a1a}-4.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.0175ms 1.3434ms 744.3974 Ops/s 753.9302 Ops/s $\color{#d91a1a}-1.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1259s 7.2789ms 137.3835 Ops/s 137.9385 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.4190ms 15.9017ms 62.8862 Ops/s 67.4605 Ops/s $\textbf{\color{#d91a1a}-6.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.7169ms 1.2677ms 788.8131 Ops/s 803.5933 Ops/s $\color{#d91a1a}-1.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1269s 9.8793ms 101.2216 Ops/s 101.5830 Ops/s $\color{#d91a1a}-0.36\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.6767ms 15.6078ms 64.0705 Ops/s 64.8677 Ops/s $\color{#d91a1a}-1.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.3762ms 1.3090ms 763.9596 Ops/s 753.9064 Ops/s $\color{#35bf28}+1.33\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot