[BugFix] Fix non-tensor passage in _StepMDP

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2260

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit 5a854e9 with merge base 39462f0 ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 41c48d3a5e82f69ceda8ddc26ea793e2fab165f7853faac056c444e2cf5f3802 /exec failed with exit code 139
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 5de872f06b6922a49cec1377fa6f3efcfc9b7de06ac7d01ecc9d57c90c9cc0ce /exec failed with exit code 1
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 638b1659bf95c770d29d7cf4bb9ed07ad52bce98de781327039be45259770c30 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 93ca15d47e48507093da8efd769987cd19aad2ca430b886635d3730d3b0edfeb /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1214s	59.9803ms	16.6722 Ops/s	18.0891 Ops/s	$\textbf{\color{#d91a1a}-7.83\%}$
test_sync	32.6320ms	30.6895ms	32.5844 Ops/s	32.4949 Ops/s	$\color{#35bf28}+0.28\%$
test_async	54.0017ms	27.9359ms	35.7962 Ops/s	34.8407 Ops/s	$\color{#35bf28}+2.74\%$
test_simple	0.3892s	0.3824s	2.6153 Ops/s	2.6525 Ops/s	$\color{#d91a1a}-1.40\%$
test_transformed	0.5404s	0.5388s	1.8558 Ops/s	1.8347 Ops/s	$\color{#35bf28}+1.15\%$
test_serial	1.3237s	1.2712s	0.7867 Ops/s	0.7860 Ops/s	$\color{#35bf28}+0.09\%$
test_parallel	1.1541s	1.1066s	0.9037 Ops/s	0.9214 Ops/s	$\color{#d91a1a}-1.93\%$
test_step_mdp_speed[True-True-True-True-True]	60.3330μs	22.0963μs	45.2564 KOps/s	44.8717 KOps/s	$\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-True-True-False]	41.7680μs	13.1203μs	76.2180 KOps/s	75.9383 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-True-True-False-True]	49.4620μs	12.8639μs	77.7368 KOps/s	77.4251 KOps/s	$\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-True-False-False]	30.7070μs	7.5518μs	132.4182 KOps/s	130.7798 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-False-True-True]	55.2130μs	23.5311μs	42.4970 KOps/s	42.1076 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-True-False]	64.5620μs	14.2223μs	70.3119 KOps/s	69.2173 KOps/s	$\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-True-False-False-True]	42.0080μs	13.9947μs	71.4557 KOps/s	70.5120 KOps/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-False-False-False]	39.0420μs	8.7589μs	114.1694 KOps/s	112.4135 KOps/s	$\color{#35bf28}+1.56\%$
test_step_mdp_speed[True-False-True-True-True]	57.2370μs	24.8765μs	40.1985 KOps/s	39.7449 KOps/s	$\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-True-True-False]	40.0650μs	15.6524μs	63.8880 KOps/s	63.8150 KOps/s	$\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-True-False-True]	37.0290μs	14.0251μs	71.3009 KOps/s	70.2692 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-False-False]	46.6970μs	8.7474μs	114.3201 KOps/s	112.4051 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-False-False-True-True]	55.5840μs	26.2003μs	38.1676 KOps/s	38.4390 KOps/s	$\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-False-False-True-False]	44.2030μs	16.7768μs	59.6061 KOps/s	59.1231 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-False-False-True]	48.7510μs	15.1787μs	65.8817 KOps/s	65.0159 KOps/s	$\color{#35bf28}+1.33\%$
test_step_mdp_speed[True-False-False-False-False]	50.9970μs	9.8769μs	101.2467 KOps/s	98.3544 KOps/s	$\color{#35bf28}+2.94\%$
test_step_mdp_speed[False-True-True-True-True]	54.2010μs	24.9330μs	40.1076 KOps/s	39.7682 KOps/s	$\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-True-True-True-False]	58.4180μs	15.6895μs	63.7369 KOps/s	63.8872 KOps/s	$\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-True-False-True]	45.7860μs	16.5976μs	60.2498 KOps/s	60.1403 KOps/s	$\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-True-True-False-False]	40.1150μs	9.9953μs	100.0466 KOps/s	97.6634 KOps/s	$\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-True-False-True-True]	56.0150μs	26.0533μs	38.3828 KOps/s	38.2578 KOps/s	$\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-False-True-False]	53.3690μs	16.7255μs	59.7891 KOps/s	59.1580 KOps/s	$\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-False-False-True]	51.4560μs	17.4499μs	57.3071 KOps/s	56.3569 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-True-False-False-False]	45.7660μs	11.1764μs	89.4746 KOps/s	88.0011 KOps/s	$\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-False-True-True-True]	54.8220μs	27.4059μs	36.4885 KOps/s	36.7097 KOps/s	$\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-False-True-True-False]	62.0970μs	17.9645μs	55.6652 KOps/s	54.6050 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[False-False-True-False-True]	45.7350μs	17.5511μs	56.9766 KOps/s	57.0550 KOps/s	$\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-False-True-False-False]	56.8260μs	11.1981μs	89.3006 KOps/s	88.5188 KOps/s	$\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-False-True-True]	40.3250μs	28.7893μs	34.7351 KOps/s	34.7942 KOps/s	$\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-False-False-True-False]	45.7550μs	19.2510μs	51.9455 KOps/s	52.1949 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-False-False-True]	42.3790μs	18.6471μs	53.6276 KOps/s	53.7471 KOps/s	$\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-False-False-False]	48.2600μs	12.3476μs	80.9872 KOps/s	80.4022 KOps/s	$\color{#35bf28}+0.73\%$
test_values[generalized_advantage_estimate-True-True]	9.6513ms	9.3785ms	106.6269 Ops/s	105.9874 Ops/s	$\color{#35bf28}+0.60\%$
test_values[vec_generalized_advantage_estimate-True-True]	36.4112ms	33.7890ms	29.5954 Ops/s	27.9500 Ops/s	$\textbf{\color{#35bf28}+5.89\%}$
test_values[td0_return_estimate-False-False]	0.2407ms	0.1659ms	6.0286 KOps/s	6.0565 KOps/s	$\color{#d91a1a}-0.46\%$
test_values[td1_return_estimate-False-False]	26.4337ms	23.4392ms	42.6635 Ops/s	41.8854 Ops/s	$\color{#35bf28}+1.86\%$
test_values[vec_td1_return_estimate-False-False]	35.7413ms	33.9207ms	29.4805 Ops/s	27.9159 Ops/s	$\textbf{\color{#35bf28}+5.60\%}$
test_values[td_lambda_return_estimate-True-False]	38.0324ms	33.7431ms	29.6357 Ops/s	28.1884 Ops/s	$\textbf{\color{#35bf28}+5.13\%}$
test_values[vec_td_lambda_return_estimate-True-False]	35.1651ms	33.8121ms	29.5752 Ops/s	27.7225 Ops/s	$\textbf{\color{#35bf28}+6.68\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.5285ms	8.3291ms	120.0613 Ops/s	118.2549 Ops/s	$\color{#35bf28}+1.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.3539ms	1.9821ms	504.5157 Ops/s	558.0242 Ops/s	$\textbf{\color{#d91a1a}-9.59\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4564ms	0.3638ms	2.7488 KOps/s	2.7475 KOps/s	$\color{#35bf28}+0.05\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	45.6141ms	44.0478ms	22.7026 Ops/s	20.7857 Ops/s	$\textbf{\color{#35bf28}+9.22\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.7003ms	3.0368ms	329.2903 Ops/s	330.6623 Ops/s	$\color{#d91a1a}-0.41\%$
test_dqn_speed	1.8660ms	1.4001ms	714.2215 Ops/s	738.3617 Ops/s	$\color{#d91a1a}-3.27\%$
test_ddpg_speed	3.6370ms	2.9161ms	342.9182 Ops/s	351.4902 Ops/s	$\color{#d91a1a}-2.44\%$
test_sac_speed	10.5058ms	8.8447ms	113.0616 Ops/s	116.0347 Ops/s	$\color{#d91a1a}-2.56\%$
test_redq_speed	15.2162ms	13.7386ms	72.7874 Ops/s	73.5075 Ops/s	$\color{#d91a1a}-0.98\%$
test_redq_deprec_speed	14.5170ms	13.4994ms	74.0773 Ops/s	68.1296 Ops/s	$\textbf{\color{#35bf28}+8.73\%}$
test_td3_speed	18.2958ms	8.7683ms	114.0472 Ops/s	117.3567 Ops/s	$\color{#d91a1a}-2.82\%$
test_cql_speed	38.5666ms	37.3062ms	26.8052 Ops/s	27.2272 Ops/s	$\color{#d91a1a}-1.55\%$
test_a2c_speed	8.0100ms	7.5131ms	133.1002 Ops/s	133.7666 Ops/s	$\color{#d91a1a}-0.50\%$
test_ppo_speed	8.9306ms	7.7948ms	128.2904 Ops/s	128.5440 Ops/s	$\color{#d91a1a}-0.20\%$
test_reinforce_speed	7.5790ms	6.7249ms	148.7016 Ops/s	150.1312 Ops/s	$\color{#d91a1a}-0.95\%$
test_iql_speed	34.4814ms	32.8478ms	30.4434 Ops/s	30.3419 Ops/s	$\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.8808ms	3.5318ms	283.1441 Ops/s	284.6696 Ops/s	$\color{#d91a1a}-0.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8737ms	0.5004ms	1.9983 KOps/s	1.7905 KOps/s	$\textbf{\color{#35bf28}+11.61\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	98.4919ms	0.5400ms	1.8518 KOps/s	2.1517 KOps/s	$\textbf{\color{#d91a1a}-13.94\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.4207ms	3.4973ms	285.9385 Ops/s	285.8151 Ops/s	$\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9791ms	0.4956ms	2.0179 KOps/s	2.0609 KOps/s	$\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8869ms	0.4676ms	2.1387 KOps/s	2.1594 KOps/s	$\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.4810ms	1.7701ms	564.9396 Ops/s	577.5065 Ops/s	$\color{#d91a1a}-2.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.4035ms	1.6667ms	599.9790 Ops/s	606.1511 Ops/s	$\color{#d91a1a}-1.02\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.1583ms	3.6293ms	275.5323 Ops/s	271.8915 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1494ms	0.6397ms	1.5633 KOps/s	1.5947 KOps/s	$\color{#d91a1a}-1.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9962ms	0.6169ms	1.6210 KOps/s	1.6642 KOps/s	$\color{#d91a1a}-2.60\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.4549ms	3.5370ms	282.7253 Ops/s	287.0542 Ops/s	$\color{#d91a1a}-1.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0349ms	0.5004ms	1.9982 KOps/s	2.0490 KOps/s	$\color{#d91a1a}-2.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7560ms	0.4794ms	2.0859 KOps/s	2.1245 KOps/s	$\color{#d91a1a}-1.81\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8196ms	3.4656ms	288.5493 Ops/s	288.4743 Ops/s	$\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5910ms	0.4915ms	2.0344 KOps/s	2.0646 KOps/s	$\color{#d91a1a}-1.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.7825ms	0.4742ms	2.1087 KOps/s	2.1446 KOps/s	$\color{#d91a1a}-1.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.3517ms	3.6256ms	275.8177 Ops/s	275.7541 Ops/s	$\color{#35bf28}+0.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.3553ms	0.6481ms	1.5429 KOps/s	1.5813 KOps/s	$\color{#d91a1a}-2.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7841ms	0.6131ms	1.6311 KOps/s	1.6513 KOps/s	$\color{#d91a1a}-1.22\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1212s	8.2776ms	120.8081 Ops/s	118.5307 Ops/s	$\color{#35bf28}+1.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	15.3004ms	12.7169ms	78.6354 Ops/s	79.7687 Ops/s	$\color{#d91a1a}-1.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.4726ms	1.1799ms	847.5388 Ops/s	948.5899 Ops/s	$\textbf{\color{#d91a1a}-10.65\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1180s	5.9692ms	167.5266 Ops/s	169.4694 Ops/s	$\color{#d91a1a}-1.15\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	14.9893ms	12.6731ms	78.9071 Ops/s	79.4269 Ops/s	$\color{#d91a1a}-0.65\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	3.8234ms	1.1069ms	903.4265 Ops/s	942.5714 Ops/s	$\color{#d91a1a}-4.15\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1121s	6.0129ms	166.3090 Ops/s	119.1843 Ops/s	$\textbf{\color{#35bf28}+39.54\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	15.3456ms	12.8765ms	77.6606 Ops/s	78.3942 Ops/s	$\color{#d91a1a}-0.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.7384ms	1.2403ms	806.2289 Ops/s	832.4584 Ops/s	$\color{#d91a1a}-3.15\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1604s	0.1069s	9.3512 Ops/s	9.7647 Ops/s	$\color{#d91a1a}-4.23\%$
test_sync	88.7251ms	87.4709ms	11.4324 Ops/s	11.1522 Ops/s	$\color{#35bf28}+2.51\%$
test_async	0.1651s	81.5658ms	12.2600 Ops/s	11.9747 Ops/s	$\color{#35bf28}+2.38\%$
test_single_pixels	0.1125s	0.1117s	8.9564 Ops/s	8.8710 Ops/s	$\color{#35bf28}+0.96\%$
test_sync_pixels	77.0606ms	70.2990ms	14.2250 Ops/s	15.0183 Ops/s	$\textbf{\color{#d91a1a}-5.28\%}$
test_async_pixels	0.1211s	55.7742ms	17.9294 Ops/s	17.8755 Ops/s	$\color{#35bf28}+0.30\%$
test_simple	0.7209s	0.7162s	1.3963 Ops/s	1.3354 Ops/s	$\color{#35bf28}+4.56\%$
test_transformed	1.0134s	0.9562s	1.0458 Ops/s	1.0298 Ops/s	$\color{#35bf28}+1.55\%$
test_serial	2.1805s	2.1183s	0.4721 Ops/s	0.4594 Ops/s	$\color{#35bf28}+2.75\%$
test_parallel	1.8830s	1.8306s	0.5463 Ops/s	0.5369 Ops/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-True-True-True]	79.8210μs	34.3286μs	29.1302 KOps/s	30.2558 KOps/s	$\color{#d91a1a}-3.72\%$
test_step_mdp_speed[True-True-True-True-False]	46.7810μs	19.8055μs	50.4909 KOps/s	51.9428 KOps/s	$\color{#d91a1a}-2.80\%$
test_step_mdp_speed[True-True-True-False-True]	43.7200μs	19.9651μs	50.0874 KOps/s	54.1194 KOps/s	$\textbf{\color{#d91a1a}-7.45\%}$
test_step_mdp_speed[True-True-True-False-False]	35.6200μs	11.5026μs	86.9366 KOps/s	91.8896 KOps/s	$\textbf{\color{#d91a1a}-5.39\%}$
test_step_mdp_speed[True-True-False-True-True]	64.7210μs	36.7405μs	27.2180 KOps/s	28.9360 KOps/s	$\textbf{\color{#d91a1a}-5.94\%}$
test_step_mdp_speed[True-True-False-True-False]	47.4610μs	21.9185μs	45.6235 KOps/s	48.1002 KOps/s	$\textbf{\color{#d91a1a}-5.15\%}$
test_step_mdp_speed[True-True-False-False-True]	52.0500μs	21.7732μs	45.9280 KOps/s	48.6942 KOps/s	$\textbf{\color{#d91a1a}-5.68\%}$
test_step_mdp_speed[True-True-False-False-False]	37.4910μs	13.3339μs	74.9967 KOps/s	78.9615 KOps/s	$\textbf{\color{#d91a1a}-5.02\%}$
test_step_mdp_speed[True-False-True-True-True]	82.3910μs	37.7845μs	26.4659 KOps/s	27.1969 KOps/s	$\color{#d91a1a}-2.69\%$
test_step_mdp_speed[True-False-True-True-False]	48.6210μs	23.7681μs	42.0731 KOps/s	43.9610 KOps/s	$\color{#d91a1a}-4.29\%$
test_step_mdp_speed[True-False-True-False-True]	46.5210μs	21.4748μs	46.5663 KOps/s	48.8558 KOps/s	$\color{#d91a1a}-4.69\%$
test_step_mdp_speed[True-False-True-False-False]	42.1800μs	13.2441μs	75.5053 KOps/s	78.5280 KOps/s	$\color{#d91a1a}-3.85\%$
test_step_mdp_speed[True-False-False-True-True]	74.0810μs	40.0875μs	24.9454 KOps/s	26.2916 KOps/s	$\textbf{\color{#d91a1a}-5.12\%}$
test_step_mdp_speed[True-False-False-True-False]	49.5610μs	25.3685μs	39.4189 KOps/s	40.8588 KOps/s	$\color{#d91a1a}-3.52\%$
test_step_mdp_speed[True-False-False-False-True]	48.1420μs	23.2761μs	42.9626 KOps/s	45.3123 KOps/s	$\textbf{\color{#d91a1a}-5.19\%}$
test_step_mdp_speed[True-False-False-False-False]	37.8400μs	15.2246μs	65.6833 KOps/s	69.7254 KOps/s	$\textbf{\color{#d91a1a}-5.80\%}$
test_step_mdp_speed[False-True-True-True-True]	69.1620μs	37.9626μs	26.3417 KOps/s	27.5215 KOps/s	$\color{#d91a1a}-4.29\%$
test_step_mdp_speed[False-True-True-True-False]	49.6110μs	23.6944μs	42.2041 KOps/s	44.1930 KOps/s	$\color{#d91a1a}-4.50\%$
test_step_mdp_speed[False-True-True-False-True]	53.2100μs	25.3855μs	39.3926 KOps/s	41.0938 KOps/s	$\color{#d91a1a}-4.14\%$
test_step_mdp_speed[False-True-True-False-False]	36.2510μs	15.1053μs	66.2018 KOps/s	69.4607 KOps/s	$\color{#d91a1a}-4.69\%$
test_step_mdp_speed[False-True-False-True-True]	66.1500μs	39.7184μs	25.1773 KOps/s	26.3047 KOps/s	$\color{#d91a1a}-4.29\%$
test_step_mdp_speed[False-True-False-True-False]	49.3300μs	25.2724μs	39.5688 KOps/s	39.9379 KOps/s	$\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-True-False-False-True]	55.5410μs	27.6557μs	36.1589 KOps/s	37.4086 KOps/s	$\color{#d91a1a}-3.34\%$
test_step_mdp_speed[False-True-False-False-False]	38.4400μs	17.0567μs	58.6280 KOps/s	59.9920 KOps/s	$\color{#d91a1a}-2.27\%$
test_step_mdp_speed[False-False-True-True-True]	69.6510μs	41.9712μs	23.8259 KOps/s	24.3101 KOps/s	$\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-False-True-True-False]	58.5220μs	27.6308μs	36.1914 KOps/s	36.5818 KOps/s	$\color{#d91a1a}-1.07\%$
test_step_mdp_speed[False-False-True-False-True]	55.0810μs	27.3429μs	36.5726 KOps/s	37.5877 KOps/s	$\color{#d91a1a}-2.70\%$
test_step_mdp_speed[False-False-True-False-False]	41.6820μs	17.0478μs	58.6586 KOps/s	60.2279 KOps/s	$\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-False-True-True]	67.8200μs	44.9638μs	22.2401 KOps/s	22.9487 KOps/s	$\color{#d91a1a}-3.09\%$
test_step_mdp_speed[False-False-False-True-False]	75.5220μs	29.4618μs	33.9423 KOps/s	33.9162 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-False-False-True]	53.0910μs	28.9158μs	34.5832 KOps/s	35.0990 KOps/s	$\color{#d91a1a}-1.47\%$
test_step_mdp_speed[False-False-False-False-False]	40.0600μs	18.5830μs	53.8127 KOps/s	54.1472 KOps/s	$\color{#d91a1a}-0.62\%$
test_values[generalized_advantage_estimate-True-True]	25.1712ms	24.8172ms	40.2946 Ops/s	39.8344 Ops/s	$\color{#35bf28}+1.16\%$
test_values[vec_generalized_advantage_estimate-True-True]	91.9210ms	2.7295ms	366.3715 Ops/s	355.9722 Ops/s	$\color{#35bf28}+2.92\%$
test_values[td0_return_estimate-False-False]	93.8710μs	64.9319μs	15.4007 KOps/s	14.4511 KOps/s	$\textbf{\color{#35bf28}+6.57\%}$
test_values[td1_return_estimate-False-False]	56.0182ms	55.2006ms	18.1158 Ops/s	17.4404 Ops/s	$\color{#35bf28}+3.87\%$
test_values[vec_td1_return_estimate-False-False]	1.4556ms	1.0698ms	934.7725 Ops/s	928.2925 Ops/s	$\color{#35bf28}+0.70\%$
test_values[td_lambda_return_estimate-True-False]	89.8521ms	88.0009ms	11.3635 Ops/s	10.8855 Ops/s	$\color{#35bf28}+4.39\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4492ms	1.0708ms	933.9035 Ops/s	920.7289 Ops/s	$\color{#35bf28}+1.43\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.7917ms	25.5490ms	39.1404 Ops/s	39.4257 Ops/s	$\color{#d91a1a}-0.72\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9485ms	0.6978ms	1.4332 KOps/s	1.3323 KOps/s	$\textbf{\color{#35bf28}+7.57\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7391ms	0.6699ms	1.4927 KOps/s	1.4328 KOps/s	$\color{#35bf28}+4.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5884ms	1.4501ms	689.6209 Ops/s	683.9510 Ops/s	$\color{#35bf28}+0.83\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7082ms	0.6653ms	1.5030 KOps/s	1.4844 KOps/s	$\color{#35bf28}+1.25\%$
test_dqn_speed	7.7089ms	1.3843ms	722.3827 Ops/s	701.0743 Ops/s	$\color{#35bf28}+3.04\%$
test_ddpg_speed	3.2379ms	2.8616ms	349.4587 Ops/s	340.9599 Ops/s	$\color{#35bf28}+2.49\%$
test_sac_speed	8.8515ms	8.2803ms	120.7682 Ops/s	120.8229 Ops/s	$\color{#d91a1a}-0.05\%$
test_redq_speed	0.1056s	11.7477ms	85.1231 Ops/s	93.2960 Ops/s	$\textbf{\color{#d91a1a}-8.76\%}$
test_redq_deprec_speed	12.0532ms	11.3650ms	87.9895 Ops/s	78.3523 Ops/s	$\textbf{\color{#35bf28}+12.30\%}$
test_td3_speed	10.0003ms	8.1762ms	122.3062 Ops/s	121.6445 Ops/s	$\color{#35bf28}+0.54\%$
test_cql_speed	26.4551ms	25.3782ms	39.4040 Ops/s	38.5148 Ops/s	$\color{#35bf28}+2.31\%$
test_a2c_speed	6.3529ms	5.6073ms	178.3381 Ops/s	174.6136 Ops/s	$\color{#35bf28}+2.13\%$
test_ppo_speed	6.3150ms	5.9572ms	167.8645 Ops/s	166.1805 Ops/s	$\color{#35bf28}+1.01\%$
test_reinforce_speed	5.4244ms	4.6064ms	217.0908 Ops/s	214.2066 Ops/s	$\color{#35bf28}+1.35\%$
test_iql_speed	19.7731ms	19.2375ms	51.9818 Ops/s	49.9158 Ops/s	$\color{#35bf28}+4.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.9354ms	4.6828ms	213.5486 Ops/s	217.0527 Ops/s	$\color{#d91a1a}-1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.5001ms	0.5423ms	1.8439 KOps/s	1.8291 KOps/s	$\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6993ms	0.5226ms	1.9134 KOps/s	1.9050 KOps/s	$\color{#35bf28}+0.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.8777ms	4.6359ms	215.7100 Ops/s	218.4147 Ops/s	$\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.5466ms	0.5363ms	1.8646 KOps/s	1.8591 KOps/s	$\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7243ms	0.5176ms	1.9321 KOps/s	1.9043 KOps/s	$\color{#35bf28}+1.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	5.8433ms	1.9832ms	504.2400 Ops/s	507.9478 Ops/s	$\color{#d91a1a}-0.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.0663ms	1.8807ms	531.7302 Ops/s	533.3100 Ops/s	$\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0507ms	4.8099ms	207.9061 Ops/s	208.4956 Ops/s	$\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.5613ms	0.6801ms	1.4704 KOps/s	1.4584 KOps/s	$\color{#35bf28}+0.83\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0777ms	0.6647ms	1.5044 KOps/s	1.4945 KOps/s	$\color{#35bf28}+0.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.8234ms	4.6566ms	214.7505 Ops/s	213.5104 Ops/s	$\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8006ms	0.5425ms	1.8434 KOps/s	1.8411 KOps/s	$\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7415ms	0.5242ms	1.9076 KOps/s	1.9131 KOps/s	$\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.9163ms	4.6195ms	216.4713 Ops/s	213.6947 Ops/s	$\color{#35bf28}+1.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7585ms	0.5403ms	1.8507 KOps/s	1.8606 KOps/s	$\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6968ms	0.5193ms	1.9257 KOps/s	1.9221 KOps/s	$\color{#35bf28}+0.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0492ms	4.8059ms	208.0790 Ops/s	205.5750 Ops/s	$\color{#35bf28}+1.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.5232ms	0.6890ms	1.4514 KOps/s	1.4575 KOps/s	$\color{#d91a1a}-0.42\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9495ms	0.6688ms	1.4952 KOps/s	1.4954 KOps/s	$\color{#d91a1a}-0.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1524s	7.9112ms	126.4023 Ops/s	136.7589 Ops/s	$\textbf{\color{#d91a1a}-7.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.2960ms	14.9820ms	66.7467 Ops/s	65.0675 Ops/s	$\color{#35bf28}+2.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.7860ms	1.3113ms	762.5929 Ops/s	894.1631 Ops/s	$\textbf{\color{#d91a1a}-14.71\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1274s	7.3744ms	135.6051 Ops/s	138.1285 Ops/s	$\color{#d91a1a}-1.83\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	17.5720ms	15.0922ms	66.2594 Ops/s	56.1534 Ops/s	$\textbf{\color{#35bf28}+18.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.5993ms	1.2518ms	798.8374 Ops/s	887.7296 Ops/s	$\textbf{\color{#d91a1a}-10.01\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1290s	10.0462ms	99.5396 Ops/s	136.2470 Ops/s	$\textbf{\color{#d91a1a}-26.94\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.5455ms	15.2425ms	65.6059 Ops/s	64.4503 Ops/s	$\color{#35bf28}+1.79\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.4979ms	1.3183ms	758.5421 Ops/s	713.2625 Ops/s	$\textbf{\color{#35bf28}+6.35\%}$