[BugFix] Fix torch 2.3 compatibility of padding indices

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2216

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Unstable CUDA signal in CI caused by cudnn 9 update

❌ 10 New Failures, 15 Unrelated Failures

As of commit 645fbba with merge base 332499a ():

NEW FAILURES - The following jobs have failed:

• Generate documentation / build-docs (3.9, 12.1) / linux-job (gh)
  No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Wheels / test-wheel (linux, ubuntu-20.04, 3.10) (gh)
  ModuleNotFoundError: No module named 'dm_env'
• Wheels / test-wheel (linux, ubuntu-20.04, 3.11) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel (linux, ubuntu-20.04, 3.8) (gh)
• Wheels / test-wheel (linux, ubuntu-20.04, 3.9) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel-windows (3.10) (gh)
• Wheels / test-wheel-windows (3.11) (gh)
  ModuleNotFoundError: No module named 'dm_env'
• Wheels / test-wheel-windows (3.8) (gh)
  ModuleNotFoundError: No module named 'dm_env'
• Wheels / test-wheel-windows (3.9) (gh)
  ModuleNotFoundError: No module named 'dm_env'

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Lint / c-source / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Lint / python-source-and-configs / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-gpu (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1076s	58.2030ms	17.1813 Ops/s	17.8856 Ops/s	$\color{#d91a1a}-3.94\%$
test_sync	35.7425ms	30.2096ms	33.1021 Ops/s	32.0326 Ops/s	$\color{#35bf28}+3.34\%$
test_async	54.1413ms	28.9729ms	34.5150 Ops/s	35.6020 Ops/s	$\color{#d91a1a}-3.05\%$
test_simple	0.4326s	0.3807s	2.6268 Ops/s	2.6715 Ops/s	$\color{#d91a1a}-1.67\%$
test_transformed	0.5319s	0.5255s	1.9030 Ops/s	1.8848 Ops/s	$\color{#35bf28}+0.97\%$
test_serial	1.2889s	1.2484s	0.8010 Ops/s	0.7937 Ops/s	$\color{#35bf28}+0.92\%$
test_parallel	1.0963s	1.0466s	0.9554 Ops/s	0.9362 Ops/s	$\color{#35bf28}+2.05\%$
test_step_mdp_speed[True-True-True-True-True]	0.1081ms	21.1710μs	47.2345 KOps/s	46.1423 KOps/s	$\color{#35bf28}+2.37\%$
test_step_mdp_speed[True-True-True-True-False]	40.2250μs	13.0603μs	76.5680 KOps/s	76.5457 KOps/s	$\color{#35bf28}+0.03\%$
test_step_mdp_speed[True-True-True-False-True]	41.0170μs	12.5034μs	79.9780 KOps/s	78.1437 KOps/s	$\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-True-True-False-False]	28.3420μs	7.5734μs	132.0413 KOps/s	129.3343 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-True-False-True-True]	59.1000μs	22.4043μs	44.6342 KOps/s	44.0922 KOps/s	$\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-False-True-False]	33.4420μs	14.1702μs	70.5705 KOps/s	69.0828 KOps/s	$\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-True-False-False-True]	52.7890μs	13.8166μs	72.3766 KOps/s	71.6366 KOps/s	$\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-True-False-False-False]	33.0610μs	8.8134μs	113.4632 KOps/s	111.8780 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-False-True-True-True]	64.7400μs	23.7563μs	42.0941 KOps/s	41.3510 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-False-True-True-False]	53.6600μs	15.5307μs	64.3887 KOps/s	62.1828 KOps/s	$\color{#35bf28}+3.55\%$
test_step_mdp_speed[True-False-True-False-True]	34.9050μs	13.7186μs	72.8938 KOps/s	71.8113 KOps/s	$\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-False-True-False-False]	36.6890μs	8.7664μs	114.0718 KOps/s	111.5430 KOps/s	$\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-False-False-True-True]	69.2690μs	24.8900μs	40.1768 KOps/s	39.7090 KOps/s	$\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-True-False]	43.0700μs	16.6035μs	60.2284 KOps/s	59.7886 KOps/s	$\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-False-False-True]	38.8830μs	14.8467μs	67.3550 KOps/s	66.7598 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-False-False-False]	39.9540μs	9.9063μs	100.9454 KOps/s	97.5493 KOps/s	$\color{#35bf28}+3.48\%$
test_step_mdp_speed[False-True-True-True-True]	58.8000μs	23.8279μs	41.9675 KOps/s	41.6465 KOps/s	$\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-True-True-True-False]	52.2370μs	15.4679μs	64.6499 KOps/s	63.6449 KOps/s	$\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-True-True-False-True]	45.9960μs	16.0676μs	62.2370 KOps/s	61.5973 KOps/s	$\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-True-True-False-False]	35.7970μs	9.9582μs	100.4196 KOps/s	97.9650 KOps/s	$\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-True-False-True-True]	53.7900μs	25.0108μs	39.9828 KOps/s	39.2594 KOps/s	$\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-True-False-True-False]	38.9020μs	16.6383μs	60.1022 KOps/s	58.7990 KOps/s	$\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-True-False-False-True]	71.8240μs	17.0932μs	58.5030 KOps/s	58.0030 KOps/s	$\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-True-False-False-False]	46.4060μs	11.1578μs	89.6231 KOps/s	87.4067 KOps/s	$\color{#35bf28}+2.54\%$
test_step_mdp_speed[False-False-True-True-True]	64.8210μs	26.2169μs	38.1433 KOps/s	37.6826 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-False-True-True-False]	54.5220μs	17.9830μs	55.6081 KOps/s	54.7960 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-False-True-False-True]	55.6440μs	17.1383μs	58.3490 KOps/s	57.5104 KOps/s	$\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-False-True-False-False]	41.2570μs	11.1222μs	89.9105 KOps/s	88.1110 KOps/s	$\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-False-False-True-True]	38.4620μs	27.6571μs	36.1571 KOps/s	35.5513 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-False-True-False]	47.3690μs	19.0186μs	52.5800 KOps/s	52.0451 KOps/s	$\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-False-False-False-True]	51.0260μs	18.0631μs	55.3615 KOps/s	54.7183 KOps/s	$\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-False-False-False-False]	40.3350μs	12.1701μs	82.1688 KOps/s	80.2055 KOps/s	$\color{#35bf28}+2.45\%$
test_values[generalized_advantage_estimate-True-True]	9.7469ms	9.4279ms	106.0680 Ops/s	106.3157 Ops/s	$\color{#d91a1a}-0.23\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.7673ms	33.2488ms	30.0763 Ops/s	28.4731 Ops/s	$\textbf{\color{#35bf28}+5.63\%}$
test_values[td0_return_estimate-False-False]	0.2145ms	0.1621ms	6.1676 KOps/s	5.6769 KOps/s	$\textbf{\color{#35bf28}+8.64\%}$
test_values[td1_return_estimate-False-False]	23.8475ms	23.6672ms	42.2525 Ops/s	42.7989 Ops/s	$\color{#d91a1a}-1.28\%$
test_values[vec_td1_return_estimate-False-False]	36.7855ms	33.2849ms	30.0436 Ops/s	28.1109 Ops/s	$\textbf{\color{#35bf28}+6.88\%}$
test_values[td_lambda_return_estimate-True-False]	37.8628ms	34.3651ms	29.0993 Ops/s	29.9000 Ops/s	$\color{#d91a1a}-2.68\%$
test_values[vec_td_lambda_return_estimate-True-False]	34.5796ms	33.1187ms	30.1944 Ops/s	28.4827 Ops/s	$\textbf{\color{#35bf28}+6.01\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.1830ms	8.3128ms	120.2966 Ops/s	120.0705 Ops/s	$\color{#35bf28}+0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	3.1933ms	2.2608ms	442.3163 Ops/s	510.6741 Ops/s	$\textbf{\color{#d91a1a}-13.39\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4365ms	0.3494ms	2.8621 KOps/s	2.8301 KOps/s	$\color{#35bf28}+1.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	40.3131ms	38.4584ms	26.0021 Ops/s	22.1582 Ops/s	$\textbf{\color{#35bf28}+17.35\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.8890ms	3.0231ms	330.7862 Ops/s	330.2192 Ops/s	$\color{#35bf28}+0.17\%$
test_dqn_speed	1.8266ms	1.3316ms	750.9877 Ops/s	733.6688 Ops/s	$\color{#35bf28}+2.36\%$
test_ddpg_speed	3.1977ms	2.8178ms	354.8818 Ops/s	350.1837 Ops/s	$\color{#35bf28}+1.34\%$
test_sac_speed	10.0947ms	8.3081ms	120.3650 Ops/s	118.5059 Ops/s	$\color{#35bf28}+1.57\%$
test_redq_speed	13.9983ms	12.9850ms	77.0122 Ops/s	75.7043 Ops/s	$\color{#35bf28}+1.73\%$
test_redq_deprec_speed	14.8417ms	13.1900ms	75.8148 Ops/s	73.8557 Ops/s	$\color{#35bf28}+2.65\%$
test_td3_speed	15.8846ms	8.3329ms	120.0063 Ops/s	118.1437 Ops/s	$\color{#35bf28}+1.58\%$
test_cql_speed	37.7127ms	36.1263ms	27.6806 Ops/s	27.3238 Ops/s	$\color{#35bf28}+1.31\%$
test_a2c_speed	7.9945ms	7.4499ms	134.2296 Ops/s	133.4383 Ops/s	$\color{#35bf28}+0.59\%$
test_ppo_speed	8.9790ms	7.7599ms	128.8684 Ops/s	127.7391 Ops/s	$\color{#35bf28}+0.88\%$
test_reinforce_speed	13.0029ms	6.8659ms	145.6480 Ops/s	147.4770 Ops/s	$\color{#d91a1a}-1.24\%$
test_iql_speed	33.8942ms	32.8891ms	30.4052 Ops/s	29.7827 Ops/s	$\color{#35bf28}+2.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.1036ms	3.4502ms	289.8411 Ops/s	273.8166 Ops/s	$\textbf{\color{#35bf28}+5.85\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8917ms	0.4952ms	2.0195 KOps/s	1.9665 KOps/s	$\color{#35bf28}+2.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6467ms	0.4784ms	2.0902 KOps/s	2.0408 KOps/s	$\color{#35bf28}+2.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.3696ms	3.5153ms	284.4687 Ops/s	272.6073 Ops/s	$\color{#35bf28}+4.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9497ms	0.4924ms	2.0310 KOps/s	2.0144 KOps/s	$\color{#35bf28}+0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7162ms	0.4698ms	2.1287 KOps/s	2.1000 KOps/s	$\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2814ms	1.7064ms	586.0334 Ops/s	586.1606 Ops/s	$\color{#d91a1a}-0.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	4.6233ms	1.6199ms	617.3062 Ops/s	614.6069 Ops/s	$\color{#35bf28}+0.44\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.2190ms	3.6198ms	276.2563 Ops/s	272.2051 Ops/s	$\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1617ms	0.6196ms	1.6140 KOps/s	1.6109 KOps/s	$\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8692ms	0.5979ms	1.6725 KOps/s	1.6902 KOps/s	$\color{#d91a1a}-1.04\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7493ms	3.6122ms	276.8374 Ops/s	290.4515 Ops/s	$\color{#d91a1a}-4.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0342ms	0.5047ms	1.9814 KOps/s	1.9999 KOps/s	$\color{#d91a1a}-0.93\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7936ms	0.4861ms	2.0570 KOps/s	2.0883 KOps/s	$\color{#d91a1a}-1.50\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8021ms	3.5352ms	282.8658 Ops/s	293.9302 Ops/s	$\color{#d91a1a}-3.76\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5850ms	0.4883ms	2.0478 KOps/s	2.0223 KOps/s	$\color{#35bf28}+1.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.5139ms	0.4742ms	2.1087 KOps/s	2.1425 KOps/s	$\color{#d91a1a}-1.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.0088ms	3.5635ms	280.6214 Ops/s	285.4375 Ops/s	$\color{#d91a1a}-1.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8438ms	0.6167ms	1.6214 KOps/s	1.6380 KOps/s	$\color{#d91a1a}-1.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7135ms	0.5860ms	1.7065 KOps/s	1.6787 KOps/s	$\color{#35bf28}+1.65\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	99.3628ms	7.4413ms	134.3843 Ops/s	129.7086 Ops/s	$\color{#35bf28}+3.60\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	14.4434ms	12.3216ms	81.1583 Ops/s	80.4506 Ops/s	$\color{#35bf28}+0.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.5292ms	1.0337ms	967.3679 Ops/s	942.3375 Ops/s	$\color{#35bf28}+2.66\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	94.8460ms	5.5867ms	178.9968 Ops/s	181.4744 Ops/s	$\color{#d91a1a}-1.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	14.3632ms	12.3399ms	81.0378 Ops/s	80.7980 Ops/s	$\color{#35bf28}+0.30\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.5274ms	1.0345ms	966.6786 Ops/s	933.9146 Ops/s	$\color{#35bf28}+3.51\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	93.4613ms	5.6265ms	177.7319 Ops/s	133.0130 Ops/s	$\textbf{\color{#35bf28}+33.62\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1060s	14.3818ms	69.5322 Ops/s	79.0266 Ops/s	$\textbf{\color{#d91a1a}-12.01\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	4.2897ms	1.2282ms	814.2041 Ops/s	846.2337 Ops/s	$\color{#d91a1a}-3.78\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1172s	0.1171s	8.5374 Ops/s	8.4202 Ops/s	$\color{#35bf28}+1.39\%$
test_sync	0.1041s	0.1026s	9.7495 Ops/s	9.6876 Ops/s	$\color{#35bf28}+0.64\%$
test_async	0.2061s	81.7523ms	12.2321 Ops/s	10.2610 Ops/s	$\textbf{\color{#35bf28}+19.21\%}$
test_single_pixels	0.1299s	0.1288s	7.7634 Ops/s	7.8076 Ops/s	$\color{#d91a1a}-0.57\%$
test_sync_pixels	83.1446ms	79.3338ms	12.6050 Ops/s	12.1286 Ops/s	$\color{#35bf28}+3.93\%$
test_async_pixels	0.1597s	69.5276ms	14.3828 Ops/s	14.4008 Ops/s	$\color{#d91a1a}-0.13\%$
test_simple	0.8908s	0.8281s	1.2076 Ops/s	1.2144 Ops/s	$\color{#d91a1a}-0.56\%$
test_transformed	1.1460s	1.0868s	0.9201 Ops/s	0.9179 Ops/s	$\color{#35bf28}+0.24\%$
test_serial	2.5774s	2.5194s	0.3969 Ops/s	0.4012 Ops/s	$\color{#d91a1a}-1.08\%$
test_parallel	2.4124s	2.3495s	0.4256 Ops/s	0.4263 Ops/s	$\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-True-True-True-True]	65.7310μs	34.0299μs	29.3859 KOps/s	30.4780 KOps/s	$\color{#d91a1a}-3.58\%$
test_step_mdp_speed[True-True-True-True-False]	39.4210μs	20.1918μs	49.5250 KOps/s	50.6261 KOps/s	$\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-True-True-False-True]	0.2821ms	19.2190μs	52.0317 KOps/s	52.5209 KOps/s	$\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-True-False-False]	32.0800μs	11.6245μs	86.0253 KOps/s	87.2900 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-True-False-True-True]	64.4310μs	35.3464μs	28.2914 KOps/s	27.9958 KOps/s	$\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-False-True-False]	41.9000μs	22.0119μs	45.4300 KOps/s	45.0247 KOps/s	$\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-True-False-False-True]	44.8310μs	21.1144μs	47.3611 KOps/s	47.5046 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-False-False]	0.2455ms	13.4386μs	74.4123 KOps/s	75.0057 KOps/s	$\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-False-True-True-True]	68.8510μs	37.3171μs	26.7974 KOps/s	26.7029 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-False-True-True-False]	48.2510μs	24.0916μs	41.5082 KOps/s	41.3187 KOps/s	$\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-False-True-False-True]	0.3057ms	21.0895μs	47.4171 KOps/s	47.5707 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-True-False-False]	0.3275ms	13.4961μs	74.0956 KOps/s	74.2233 KOps/s	$\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-False-False-True-True]	62.9010μs	39.0083μs	25.6356 KOps/s	25.2977 KOps/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-False-False-True-False]	51.3610μs	26.0890μs	38.3304 KOps/s	39.1540 KOps/s	$\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-False-False-False-True]	48.9310μs	22.8278μs	43.8062 KOps/s	43.6879 KOps/s	$\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-False-False-False-False]	32.1410μs	15.2560μs	65.5481 KOps/s	66.0688 KOps/s	$\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-True-True-True]	59.9920μs	37.4868μs	26.6760 KOps/s	26.7534 KOps/s	$\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-True-True-True-False]	43.9110μs	24.1762μs	41.3630 KOps/s	41.9104 KOps/s	$\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-True-True-False-True]	42.3510μs	25.3379μs	39.4666 KOps/s	39.8743 KOps/s	$\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-True-True-False-False]	41.5310μs	15.4675μs	64.6516 KOps/s	64.4936 KOps/s	$\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-True-False-True-True]	61.4320μs	39.2199μs	25.4973 KOps/s	25.4457 KOps/s	$\color{#35bf28}+0.20\%$
test_step_mdp_speed[False-True-False-True-False]	46.6510μs	25.9307μs	38.5644 KOps/s	39.3753 KOps/s	$\color{#d91a1a}-2.06\%$
test_step_mdp_speed[False-True-False-False-True]	42.9510μs	27.1860μs	36.7836 KOps/s	38.0609 KOps/s	$\color{#d91a1a}-3.36\%$
test_step_mdp_speed[False-True-False-False-False]	37.1800μs	17.1169μs	58.4218 KOps/s	58.7607 KOps/s	$\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-True-True-True]	78.6020μs	41.1045μs	24.3283 KOps/s	24.3743 KOps/s	$\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-False-True-True-False]	52.1010μs	27.9720μs	35.7501 KOps/s	35.9066 KOps/s	$\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-False-True-False-True]	50.1610μs	27.3527μs	36.5594 KOps/s	37.3651 KOps/s	$\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-False-True-False-False]	44.0310μs	17.2128μs	58.0963 KOps/s	57.8578 KOps/s	$\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-False-True-True]	78.8310μs	44.5336μs	22.4549 KOps/s	22.4235 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-False-False-True-False]	59.9010μs	29.9210μs	33.4213 KOps/s	33.7450 KOps/s	$\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-False-False-False-True]	53.4810μs	28.9566μs	34.5344 KOps/s	35.3098 KOps/s	$\color{#d91a1a}-2.20\%$
test_step_mdp_speed[False-False-False-False-False]	44.4810μs	18.8236μs	53.1249 KOps/s	52.2653 KOps/s	$\color{#35bf28}+1.64\%$
test_values[generalized_advantage_estimate-True-True]	24.9399ms	24.1626ms	41.3863 Ops/s	41.2914 Ops/s	$\color{#35bf28}+0.23\%$
test_values[vec_generalized_advantage_estimate-True-True]	92.8437ms	2.7630ms	361.9235 Ops/s	368.7191 Ops/s	$\color{#d91a1a}-1.84\%$
test_values[td0_return_estimate-False-False]	94.4120μs	67.9926μs	14.7075 KOps/s	14.9150 KOps/s	$\color{#d91a1a}-1.39\%$
test_values[td1_return_estimate-False-False]	57.4611ms	56.3707ms	17.7397 Ops/s	18.2744 Ops/s	$\color{#d91a1a}-2.93\%$
test_values[vec_td1_return_estimate-False-False]	1.3935ms	1.0935ms	914.4863 Ops/s	915.1850 Ops/s	$\color{#d91a1a}-0.08\%$
test_values[td_lambda_return_estimate-True-False]	92.5002ms	90.2882ms	11.0756 Ops/s	11.5151 Ops/s	$\color{#d91a1a}-3.82\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4890ms	1.1001ms	909.0057 Ops/s	913.8311 Ops/s	$\color{#d91a1a}-0.53\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.7896ms	24.7791ms	40.3565 Ops/s	41.0244 Ops/s	$\color{#d91a1a}-1.63\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9485ms	0.7255ms	1.3783 KOps/s	1.3436 KOps/s	$\color{#35bf28}+2.58\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.9043ms	0.6780ms	1.4749 KOps/s	1.4723 KOps/s	$\color{#35bf28}+0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.7862ms	1.4786ms	676.3006 Ops/s	678.5258 Ops/s	$\color{#d91a1a}-0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.0054ms	0.6946ms	1.4397 KOps/s	1.4390 KOps/s	$\color{#35bf28}+0.05\%$
test_dqn_speed	9.1611ms	1.5204ms	657.7392 Ops/s	689.4284 Ops/s	$\color{#d91a1a}-4.60\%$
test_ddpg_speed	3.4282ms	3.0459ms	328.3151 Ops/s	332.1401 Ops/s	$\color{#d91a1a}-1.15\%$
test_sac_speed	8.8981ms	8.5386ms	117.1146 Ops/s	119.4085 Ops/s	$\color{#d91a1a}-1.92\%$
test_redq_speed	13.0917ms	10.9475ms	91.3452 Ops/s	84.2605 Ops/s	$\textbf{\color{#35bf28}+8.41\%}$
test_redq_deprec_speed	12.1988ms	11.7589ms	85.0418 Ops/s	83.5111 Ops/s	$\color{#35bf28}+1.83\%$
test_td3_speed	17.3314ms	8.4952ms	117.7133 Ops/s	121.2375 Ops/s	$\color{#d91a1a}-2.91\%$
test_cql_speed	27.3160ms	26.1791ms	38.1985 Ops/s	38.6267 Ops/s	$\color{#d91a1a}-1.11\%$
test_a2c_speed	6.5518ms	5.6892ms	175.7725 Ops/s	171.7228 Ops/s	$\color{#35bf28}+2.36\%$
test_ppo_speed	6.3848ms	6.0000ms	166.6662 Ops/s	163.1451 Ops/s	$\color{#35bf28}+2.16\%$
test_reinforce_speed	5.0192ms	4.6587ms	214.6504 Ops/s	212.3183 Ops/s	$\color{#35bf28}+1.10\%$
test_iql_speed	21.5072ms	20.3634ms	49.1077 Ops/s	50.3223 Ops/s	$\color{#d91a1a}-2.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.0976ms	4.8235ms	207.3189 Ops/s	215.1717 Ops/s	$\color{#d91a1a}-3.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7370ms	0.5958ms	1.6785 KOps/s	1.6721 KOps/s	$\color{#35bf28}+0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.4397ms	0.5755ms	1.7375 KOps/s	1.7411 KOps/s	$\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0333ms	4.7648ms	209.8719 Ops/s	214.5074 Ops/s	$\color{#d91a1a}-2.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9761ms	0.5910ms	1.6920 KOps/s	1.6953 KOps/s	$\color{#d91a1a}-0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.3890ms	0.5692ms	1.7568 KOps/s	1.7512 KOps/s	$\color{#35bf28}+0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.4056ms	2.0550ms	486.6154 Ops/s	485.3422 Ops/s	$\color{#35bf28}+0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	5.8501ms	1.9781ms	505.5257 Ops/s	513.7287 Ops/s	$\color{#d91a1a}-1.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.1415ms	4.9127ms	203.5542 Ops/s	209.5485 Ops/s	$\color{#d91a1a}-2.86\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.5482ms	0.7371ms	1.3567 KOps/s	1.3880 KOps/s	$\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0726ms	0.7095ms	1.4094 KOps/s	1.4313 KOps/s	$\color{#d91a1a}-1.53\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.9948ms	4.7746ms	209.4414 Ops/s	215.1377 Ops/s	$\color{#d91a1a}-2.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.2796ms	0.6014ms	1.6628 KOps/s	1.6818 KOps/s	$\color{#d91a1a}-1.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7292ms	0.5794ms	1.7260 KOps/s	1.7298 KOps/s	$\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.9661ms	4.7189ms	211.9134 Ops/s	214.4318 Ops/s	$\color{#d91a1a}-1.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7421ms	0.5939ms	1.6838 KOps/s	1.6979 KOps/s	$\color{#d91a1a}-0.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7003ms	0.5701ms	1.7539 KOps/s	1.7633 KOps/s	$\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0671ms	4.8928ms	204.3833 Ops/s	208.7025 Ops/s	$\color{#d91a1a}-2.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.3948ms	0.7274ms	1.3747 KOps/s	1.3807 KOps/s	$\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0728ms	0.7079ms	1.4126 KOps/s	1.4165 KOps/s	$\color{#d91a1a}-0.28\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1334s	7.5638ms	132.2095 Ops/s	131.0945 Ops/s	$\color{#35bf28}+0.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.4022ms	15.7765ms	63.3856 Ops/s	64.3731 Ops/s	$\color{#d91a1a}-1.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.5129ms	1.5195ms	658.1064 Ops/s	749.9060 Ops/s	$\textbf{\color{#d91a1a}-12.24\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1207s	7.3251ms	136.5167 Ops/s	104.3602 Ops/s	$\textbf{\color{#35bf28}+30.81\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.3573ms	15.8298ms	63.1720 Ops/s	64.7863 Ops/s	$\color{#d91a1a}-2.49\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.3392ms	1.3269ms	753.6219 Ops/s	741.6394 Ops/s	$\color{#35bf28}+1.62\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1217s	9.8153ms	101.8817 Ops/s	136.2396 Ops/s	$\textbf{\color{#d91a1a}-25.22\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.4225ms	15.8818ms	62.9650 Ops/s	64.2138 Ops/s	$\color{#d91a1a}-1.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.1185ms	1.6079ms	621.9117 Ops/s	666.8354 Ops/s	$\textbf{\color{#d91a1a}-6.74\%}$