[Doc] Getting started tutos

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1886

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (5 Unrelated Failures)

As of commit 398e561 with merge base 4d52d5f ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 53ca6b533cd22fc8b20eb939c9fbe5ccc320509778560f4ee300e15827e64e25 /exec failed with exit code 139
• Lint / c-source / linux-job (gh)
  RuntimeError: Command docker exec -t 488124b1499ceada44ea8d601e700c57c2b4f2c5361bf7ea16a1d1801364126d /exec failed with exit code 1
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1566s	68.9707ms	14.4989 Ops/s	15.4686 Ops/s	$\textbf{\color{#d91a1a}-6.27\%}$
test_sync	42.3870ms	38.5610ms	25.9329 Ops/s	25.5467 Ops/s	$\color{#35bf28}+1.51\%$
test_async	0.1112s	33.4393ms	29.9049 Ops/s	29.1259 Ops/s	$\color{#35bf28}+2.67\%$
test_simple	0.5264s	0.4440s	2.2520 Ops/s	2.2093 Ops/s	$\color{#35bf28}+1.93\%$
test_transformed	0.6895s	0.6102s	1.6388 Ops/s	1.6103 Ops/s	$\color{#35bf28}+1.77\%$
test_serial	1.5137s	1.4478s	0.6907 Ops/s	0.6780 Ops/s	$\color{#35bf28}+1.86\%$
test_parallel	1.5580s	1.4382s	0.6953 Ops/s	0.7015 Ops/s	$\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-True-True-True]	0.1805ms	21.1545μs	47.2712 KOps/s	46.4464 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-True-True-True-False]	44.0520μs	12.9459μs	77.2444 KOps/s	76.3822 KOps/s	$\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-True-True-False-True]	33.4830μs	12.4353μs	80.4160 KOps/s	78.4359 KOps/s	$\color{#35bf28}+2.52\%$
test_step_mdp_speed[True-True-True-False-False]	45.7450μs	7.4757μs	133.7670 KOps/s	131.4619 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-False-True-True]	54.6820μs	22.4335μs	44.5761 KOps/s	43.2159 KOps/s	$\color{#35bf28}+3.15\%$
test_step_mdp_speed[True-True-False-True-False]	43.0600μs	14.1951μs	70.4468 KOps/s	69.1856 KOps/s	$\color{#35bf28}+1.82\%$
test_step_mdp_speed[True-True-False-False-True]	86.4510μs	13.6805μs	73.0965 KOps/s	72.1980 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-False-False-False]	54.7420μs	8.9559μs	111.6582 KOps/s	111.8697 KOps/s	$\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-False-True-True-True]	64.0890μs	24.0440μs	41.5904 KOps/s	41.0783 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-True-True-False]	46.3160μs	15.5797μs	64.1861 KOps/s	65.0683 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-False-True]	39.9250μs	13.7123μs	72.9273 KOps/s	72.8763 KOps/s	$\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-True-False-False]	40.3250μs	8.7943μs	113.7098 KOps/s	113.8225 KOps/s	$\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-False-False-True-True]	49.3620μs	25.2051μs	39.6746 KOps/s	39.8122 KOps/s	$\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-False-False-True-False]	44.5230μs	16.4718μs	60.7099 KOps/s	59.2296 KOps/s	$\color{#35bf28}+2.50\%$
test_step_mdp_speed[True-False-False-False-True]	39.8240μs	14.7081μs	67.9898 KOps/s	66.7731 KOps/s	$\color{#35bf28}+1.82\%$
test_step_mdp_speed[True-False-False-False-False]	45.4950μs	9.7951μs	102.0921 KOps/s	98.7763 KOps/s	$\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-True-True-True-True]	53.3300μs	23.6498μs	42.2837 KOps/s	41.3723 KOps/s	$\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-True-True-True-False]	70.1710μs	15.4714μs	64.6353 KOps/s	63.3986 KOps/s	$\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-True-False-True]	60.9540μs	15.8246μs	63.1926 KOps/s	61.9694 KOps/s	$\color{#35bf28}+1.97\%$
test_step_mdp_speed[False-True-True-False-False]	49.3520μs	9.9994μs	100.0059 KOps/s	99.2677 KOps/s	$\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-True-False-True-True]	39.9340μs	26.2358μs	38.1158 KOps/s	38.1849 KOps/s	$\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-False-True-False]	56.1950μs	16.7977μs	59.5319 KOps/s	58.3134 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-True-False-False-True]	53.4290μs	16.9467μs	59.0086 KOps/s	58.2231 KOps/s	$\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-False-False-False]	56.5660μs	11.0810μs	90.2449 KOps/s	88.1672 KOps/s	$\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-False-True-True-True]	60.6330μs	26.2594μs	38.0816 KOps/s	37.5931 KOps/s	$\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-True-True-False]	47.5790μs	18.1708μs	55.0334 KOps/s	54.6251 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-True-False-True]	38.6320μs	17.0980μs	58.4863 KOps/s	57.5148 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-False-True-False-False]	44.0820μs	11.1715μs	89.5134 KOps/s	87.8233 KOps/s	$\color{#35bf28}+1.92\%$
test_step_mdp_speed[False-False-False-True-True]	66.4640μs	27.1464μs	36.8373 KOps/s	36.1820 KOps/s	$\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-False-False-True-False]	84.7580μs	19.0108μs	52.6016 KOps/s	51.4402 KOps/s	$\color{#35bf28}+2.26\%$
test_step_mdp_speed[False-False-False-False-True]	60.9140μs	18.2209μs	54.8819 KOps/s	54.2725 KOps/s	$\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-False-False-False-False]	68.4470μs	12.1728μs	82.1507 KOps/s	79.3605 KOps/s	$\color{#35bf28}+3.52\%$
test_values[generalized_advantage_estimate-True-True]	10.5136ms	9.6022ms	104.1426 Ops/s	109.9145 Ops/s	$\textbf{\color{#d91a1a}-5.25\%}$
test_values[vec_generalized_advantage_estimate-True-True]	41.5458ms	36.3133ms	27.5381 Ops/s	28.5827 Ops/s	$\color{#d91a1a}-3.65\%$
test_values[td0_return_estimate-False-False]	0.2538ms	0.1729ms	5.7822 KOps/s	5.8520 KOps/s	$\color{#d91a1a}-1.19\%$
test_values[td1_return_estimate-False-False]	23.6986ms	23.0793ms	43.3288 Ops/s	42.9408 Ops/s	$\color{#35bf28}+0.90\%$
test_values[vec_td1_return_estimate-False-False]	42.5193ms	37.1682ms	26.9047 Ops/s	28.5166 Ops/s	$\textbf{\color{#d91a1a}-5.65\%}$
test_values[td_lambda_return_estimate-True-False]	38.3858ms	33.7415ms	29.6371 Ops/s	29.9449 Ops/s	$\color{#d91a1a}-1.03\%$
test_values[vec_td_lambda_return_estimate-True-False]	40.6763ms	36.8972ms	27.1023 Ops/s	28.4305 Ops/s	$\color{#d91a1a}-4.67\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.6597ms	8.0609ms	124.0564 Ops/s	124.9876 Ops/s	$\color{#d91a1a}-0.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.6185ms	2.0417ms	489.7944 Ops/s	487.7339 Ops/s	$\color{#35bf28}+0.42\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5500ms	0.3596ms	2.7806 KOps/s	2.7818 KOps/s	$\color{#d91a1a}-0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	54.2481ms	48.3406ms	20.6866 Ops/s	20.9024 Ops/s	$\color{#d91a1a}-1.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.7170ms	3.0279ms	330.2624 Ops/s	329.8463 Ops/s	$\color{#35bf28}+0.13\%$
test_dqn_speed	95.9954ms	1.5566ms	642.4163 Ops/s	640.7004 Ops/s	$\color{#35bf28}+0.27\%$
test_ddpg_speed	3.5799ms	2.7832ms	359.3043 Ops/s	356.0273 Ops/s	$\color{#35bf28}+0.92\%$
test_sac_speed	9.2867ms	8.7698ms	114.0274 Ops/s	114.0546 Ops/s	$\color{#d91a1a}-0.02\%$
test_redq_speed	18.2486ms	15.1370ms	66.0635 Ops/s	68.1986 Ops/s	$\color{#d91a1a}-3.13\%$
test_redq_deprec_speed	17.3215ms	14.8182ms	67.4844 Ops/s	70.2226 Ops/s	$\color{#d91a1a}-3.90\%$
test_td3_speed	10.3910ms	9.0285ms	110.7609 Ops/s	111.9378 Ops/s	$\color{#d91a1a}-1.05\%$
test_cql_speed	44.4812ms	39.7097ms	25.1828 Ops/s	25.8949 Ops/s	$\color{#d91a1a}-2.75\%$
test_a2c_speed	10.1494ms	7.9163ms	126.3221 Ops/s	132.9407 Ops/s	$\color{#d91a1a}-4.98\%$
test_ppo_speed	15.7221ms	9.5100ms	105.1522 Ops/s	129.8032 Ops/s	$\textbf{\color{#d91a1a}-18.99\%}$
test_reinforce_speed	10.0375ms	7.8396ms	127.5571 Ops/s	149.3963 Ops/s	$\textbf{\color{#d91a1a}-14.62\%}$
test_iql_speed	39.7575ms	37.2107ms	26.8740 Ops/s	29.4654 Ops/s	$\textbf{\color{#d91a1a}-8.79\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.2593ms	3.4889ms	286.6252 Ops/s	350.6880 Ops/s	$\textbf{\color{#d91a1a}-18.27\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.9506ms	0.5735ms	1.7438 KOps/s	1.8766 KOps/s	$\textbf{\color{#d91a1a}-7.08\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8593ms	0.5397ms	1.8529 KOps/s	1.9083 KOps/s	$\color{#d91a1a}-2.90\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.2488ms	3.4326ms	291.3257 Ops/s	340.5282 Ops/s	$\textbf{\color{#d91a1a}-14.45\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7562ms	0.5386ms	1.8568 KOps/s	1.8923 KOps/s	$\color{#d91a1a}-1.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8363ms	0.5094ms	1.9632 KOps/s	1.9887 KOps/s	$\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.8406ms	3.1960ms	312.8898 Ops/s	329.1058 Ops/s	$\color{#d91a1a}-4.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0158ms	0.6520ms	1.5338 KOps/s	1.5187 KOps/s	$\color{#35bf28}+0.99\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9327ms	0.6105ms	1.6379 KOps/s	1.5715 KOps/s	$\color{#35bf28}+4.23\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.9956ms	2.9484ms	339.1674 Ops/s	336.9134 Ops/s	$\color{#35bf28}+0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7015ms	0.5242ms	1.9075 KOps/s	1.8652 KOps/s	$\color{#35bf28}+2.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6191ms	0.4916ms	2.0340 KOps/s	1.9609 KOps/s	$\color{#35bf28}+3.73\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.1310ms	3.0860ms	324.0466 Ops/s	321.1353 Ops/s	$\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6548ms	0.5189ms	1.9270 KOps/s	1.8703 KOps/s	$\color{#35bf28}+3.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6505ms	0.4833ms	2.0693 KOps/s	1.9836 KOps/s	$\color{#35bf28}+4.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.7870ms	3.0270ms	330.3573 Ops/s	313.1300 Ops/s	$\textbf{\color{#35bf28}+5.50\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8357ms	0.6444ms	1.5519 KOps/s	1.5057 KOps/s	$\color{#35bf28}+3.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9135ms	0.6146ms	1.6271 KOps/s	1.6086 KOps/s	$\color{#35bf28}+1.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1767s	10.3072ms	97.0200 Ops/s	104.1550 Ops/s	$\textbf{\color{#d91a1a}-6.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.5749ms	13.8198ms	72.3598 Ops/s	72.9781 Ops/s	$\color{#d91a1a}-0.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.2110ms	2.6371ms	379.1981 Ops/s	374.8018 Ops/s	$\color{#35bf28}+1.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1223s	11.2123ms	89.1877 Ops/s	111.8481 Ops/s	$\textbf{\color{#d91a1a}-20.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.1510ms	13.5914ms	73.5759 Ops/s	58.4469 Ops/s	$\textbf{\color{#35bf28}+25.88\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.9145ms	2.6462ms	377.8936 Ops/s	372.0992 Ops/s	$\color{#35bf28}+1.56\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.2052s	10.8482ms	92.1815 Ops/s	88.6705 Ops/s	$\color{#35bf28}+3.96\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1319s	16.3758ms	61.0657 Ops/s	71.8328 Ops/s	$\textbf{\color{#d91a1a}-14.99\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	5.5689ms	2.9962ms	333.7606 Ops/s	336.1138 Ops/s	$\color{#d91a1a}-0.70\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1185s	0.1146s	8.7250 Ops/s	8.7012 Ops/s	$\color{#35bf28}+0.27\%$
test_sync	0.1700s	0.1027s	9.7361 Ops/s	9.7279 Ops/s	$\color{#35bf28}+0.08\%$
test_async	0.2489s	92.1445ms	10.8525 Ops/s	10.8564 Ops/s	$\color{#d91a1a}-0.04\%$
test_single_pixels	0.1261s	0.1259s	7.9457 Ops/s	7.6451 Ops/s	$\color{#35bf28}+3.93\%$
test_sync_pixels	90.1847ms	83.1176ms	12.0311 Ops/s	12.6305 Ops/s	$\color{#d91a1a}-4.75\%$
test_async_pixels	0.2221s	76.1828ms	13.1263 Ops/s	13.2275 Ops/s	$\color{#d91a1a}-0.77\%$
test_simple	0.8919s	0.8161s	1.2253 Ops/s	1.2375 Ops/s	$\color{#d91a1a}-0.99\%$
test_transformed	1.1243s	1.0619s	0.9417 Ops/s	0.9316 Ops/s	$\color{#35bf28}+1.09\%$
test_serial	2.4426s	2.3617s	0.4234 Ops/s	0.4200 Ops/s	$\color{#35bf28}+0.82\%$
test_parallel	2.2370s	2.0705s	0.4830 Ops/s	0.4893 Ops/s	$\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-True-True-True-True]	85.7610μs	31.9392μs	31.3095 KOps/s	30.4637 KOps/s	$\color{#35bf28}+2.78\%$
test_step_mdp_speed[True-True-True-True-False]	41.8120μs	19.2487μs	51.9515 KOps/s	52.0192 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-False-True]	37.3100μs	18.3202μs	54.5845 KOps/s	54.0595 KOps/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-True-True-False-False]	34.1910μs	10.9256μs	91.5283 KOps/s	90.7530 KOps/s	$\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-True-True]	57.4210μs	33.6957μs	29.6774 KOps/s	29.5348 KOps/s	$\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-True-False-True-False]	38.6310μs	20.9180μs	47.8056 KOps/s	47.8262 KOps/s	$\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-False-False-True]	50.2110μs	20.0772μs	49.8078 KOps/s	49.4607 KOps/s	$\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-True-False-False-False]	30.6610μs	12.8745μs	77.6728 KOps/s	78.2489 KOps/s	$\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-True-True]	56.7620μs	35.8638μs	27.8833 KOps/s	27.5621 KOps/s	$\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-True-False]	46.5410μs	22.7280μs	43.9985 KOps/s	43.9087 KOps/s	$\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-False-True-False-True]	41.6300μs	19.8594μs	50.3540 KOps/s	48.8401 KOps/s	$\color{#35bf28}+3.10\%$
test_step_mdp_speed[True-False-True-False-False]	31.0010μs	12.7942μs	78.1601 KOps/s	78.5464 KOps/s	$\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-False-False-True-True]	66.1010μs	37.4141μs	26.7279 KOps/s	26.5240 KOps/s	$\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-False-False-True-False]	47.0710μs	24.5181μs	40.7862 KOps/s	40.9158 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-False-False-True]	45.7210μs	21.6641μs	46.1594 KOps/s	45.0546 KOps/s	$\color{#35bf28}+2.45\%$
test_step_mdp_speed[True-False-False-False-False]	40.5610μs	14.6199μs	68.3998 KOps/s	68.8522 KOps/s	$\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-True-True-True-True]	53.2600μs	35.8183μs	27.9187 KOps/s	27.4737 KOps/s	$\color{#35bf28}+1.62\%$
test_step_mdp_speed[False-True-True-True-False]	51.3220μs	22.8560μs	43.7523 KOps/s	44.0251 KOps/s	$\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-True-True-False-True]	45.3900μs	23.7969μs	42.0222 KOps/s	41.5187 KOps/s	$\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-True-True-False-False]	32.5600μs	14.4372μs	69.2653 KOps/s	68.8625 KOps/s	$\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-True-False-True-True]	82.6420μs	37.7916μs	26.4609 KOps/s	25.9752 KOps/s	$\color{#35bf28}+1.87\%$
test_step_mdp_speed[False-True-False-True-False]	45.5410μs	25.0528μs	39.9157 KOps/s	40.5049 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-True-False-False-True]	42.3220μs	25.1232μs	39.8039 KOps/s	38.8137 KOps/s	$\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-True-False-False-False]	37.6200μs	16.3695μs	61.0893 KOps/s	60.8275 KOps/s	$\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-True-True-True]	62.6610μs	39.3043μs	25.4425 KOps/s	25.1293 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-True-True-False]	52.9100μs	26.6940μs	37.4616 KOps/s	37.5956 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-True-False-True]	45.1600μs	25.2098μs	39.6672 KOps/s	38.5312 KOps/s	$\color{#35bf28}+2.95\%$
test_step_mdp_speed[False-False-True-False-False]	36.4310μs	16.2564μs	61.5143 KOps/s	61.4155 KOps/s	$\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-False-False-True-True]	70.7710μs	40.5782μs	24.6438 KOps/s	24.4333 KOps/s	$\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-True-False]	47.2010μs	28.0722μs	35.6224 KOps/s	35.6409 KOps/s	$\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-False-False-False-True]	53.5510μs	26.9530μs	37.1016 KOps/s	37.0446 KOps/s	$\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-False-False-False]	36.4400μs	17.9862μs	55.5981 KOps/s	55.7670 KOps/s	$\color{#d91a1a}-0.30\%$
test_values[generalized_advantage_estimate-True-True]	24.7989ms	24.1466ms	41.4137 Ops/s	42.3215 Ops/s	$\color{#d91a1a}-2.15\%$
test_values[vec_generalized_advantage_estimate-True-True]	81.8672ms	3.1906ms	313.4160 Ops/s	309.0894 Ops/s	$\color{#35bf28}+1.40\%$
test_values[td0_return_estimate-False-False]	98.3620μs	59.8021μs	16.7218 KOps/s	16.8052 KOps/s	$\color{#d91a1a}-0.50\%$
test_values[td1_return_estimate-False-False]	52.3061ms	51.7943ms	19.3071 Ops/s	18.9474 Ops/s	$\color{#35bf28}+1.90\%$
test_values[vec_td1_return_estimate-False-False]	2.0454ms	1.7466ms	572.5522 Ops/s	571.6189 Ops/s	$\color{#35bf28}+0.16\%$
test_values[td_lambda_return_estimate-True-False]	83.5614ms	82.7900ms	12.0787 Ops/s	12.2310 Ops/s	$\color{#d91a1a}-1.24\%$
test_values[vec_td_lambda_return_estimate-True-False]	4.0175ms	1.7852ms	560.1706 Ops/s	560.5445 Ops/s	$\color{#d91a1a}-0.07\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	23.3971ms	23.1368ms	43.2213 Ops/s	44.2307 Ops/s	$\color{#d91a1a}-2.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8829ms	0.6909ms	1.4474 KOps/s	1.4508 KOps/s	$\color{#d91a1a}-0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.6951ms	0.6430ms	1.5551 KOps/s	1.5201 KOps/s	$\color{#35bf28}+2.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4719ms	1.4430ms	693.0228 Ops/s	694.3480 Ops/s	$\color{#d91a1a}-0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9244ms	0.6604ms	1.5143 KOps/s	1.5176 KOps/s	$\color{#d91a1a}-0.22\%$
test_dqn_speed	8.0566ms	1.4227ms	702.9079 Ops/s	710.0628 Ops/s	$\color{#d91a1a}-1.01\%$
test_ddpg_speed	3.1873ms	2.7255ms	366.9066 Ops/s	370.2166 Ops/s	$\color{#d91a1a}-0.89\%$
test_sac_speed	8.7144ms	8.3446ms	119.8374 Ops/s	120.2928 Ops/s	$\color{#d91a1a}-0.38\%$
test_redq_speed	12.7316ms	10.4768ms	95.4489 Ops/s	96.6462 Ops/s	$\color{#d91a1a}-1.24\%$
test_redq_deprec_speed	11.9537ms	11.4838ms	87.0789 Ops/s	86.6456 Ops/s	$\color{#35bf28}+0.50\%$
test_td3_speed	17.5962ms	8.5183ms	117.3948 Ops/s	117.1100 Ops/s	$\color{#35bf28}+0.24\%$
test_cql_speed	26.4565ms	25.6310ms	39.0153 Ops/s	39.3352 Ops/s	$\color{#d91a1a}-0.81\%$
test_a2c_speed	5.7348ms	5.5001ms	181.8150 Ops/s	182.8060 Ops/s	$\color{#d91a1a}-0.54\%$
test_ppo_speed	6.2429ms	5.7461ms	174.0325 Ops/s	174.7721 Ops/s	$\color{#d91a1a}-0.42\%$
test_reinforce_speed	5.2060ms	4.4748ms	223.4736 Ops/s	225.5604 Ops/s	$\color{#d91a1a}-0.93\%$
test_iql_speed	19.8729ms	19.4945ms	51.2966 Ops/s	50.9353 Ops/s	$\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.6560ms	3.5317ms	283.1487 Ops/s	281.4339 Ops/s	$\color{#35bf28}+0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7300ms	0.5552ms	1.8010 KOps/s	1.7966 KOps/s	$\color{#35bf28}+0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7300ms	0.5295ms	1.8887 KOps/s	1.8960 KOps/s	$\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.7772ms	3.5630ms	280.6625 Ops/s	279.9124 Ops/s	$\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7134ms	0.5490ms	1.8214 KOps/s	1.8211 KOps/s	$\color{#35bf28}+0.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7456ms	0.5244ms	1.9070 KOps/s	1.9081 KOps/s	$\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.7733ms	3.6847ms	271.3921 Ops/s	273.4587 Ops/s	$\color{#d91a1a}-0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9297ms	0.6880ms	1.4534 KOps/s	1.4728 KOps/s	$\color{#d91a1a}-1.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8108ms	0.6563ms	1.5238 KOps/s	1.5371 KOps/s	$\color{#d91a1a}-0.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7083ms	3.5338ms	282.9832 Ops/s	280.7918 Ops/s	$\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7355ms	0.5565ms	1.7970 KOps/s	1.7927 KOps/s	$\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7497ms	0.5303ms	1.8858 KOps/s	1.8846 KOps/s	$\color{#35bf28}+0.07\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8202ms	3.5937ms	278.2681 Ops/s	278.3318 Ops/s	$\color{#d91a1a}-0.02\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7172ms	0.5512ms	1.8144 KOps/s	1.8142 KOps/s	$\color{#35bf28}+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6547ms	0.5251ms	1.9045 KOps/s	1.9049 KOps/s	$\color{#d91a1a}-0.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.8634ms	3.7110ms	269.4656 Ops/s	269.5609 Ops/s	$\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9066ms	0.6869ms	1.4559 KOps/s	1.4597 KOps/s	$\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8847ms	0.6594ms	1.5166 KOps/s	1.5126 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1250s	10.0957ms	99.0523 Ops/s	98.2651 Ops/s	$\color{#35bf28}+0.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1274s	18.3600ms	54.4661 Ops/s	60.9224 Ops/s	$\textbf{\color{#d91a1a}-10.60\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.5515ms	3.0727ms	325.4464 Ops/s	324.8151 Ops/s	$\color{#35bf28}+0.19\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1186s	9.9523ms	100.4790 Ops/s	99.5891 Ops/s	$\color{#35bf28}+0.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.5675ms	16.1392ms	61.9609 Ops/s	53.4215 Ops/s	$\textbf{\color{#35bf28}+15.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.4868ms	3.0936ms	323.2532 Ops/s	320.9711 Ops/s	$\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1218s	10.3216ms	96.8841 Ops/s	97.0696 Ops/s	$\color{#d91a1a}-0.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.7929ms	16.4181ms	60.9085 Ops/s	59.8276 Ops/s	$\color{#35bf28}+1.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.4302ms	3.3827ms	295.6220 Ops/s	292.8409 Ops/s	$\color{#35bf28}+0.95\%$