[BugFix] Fix slice sampler compatibility with split_trajs and MultiStep

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1961

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit beb4f0f with merge base 7782751 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  Process completed with exit code 2.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1159s	0.1131s	8.8443 Ops/s	8.4502 Ops/s	$\color{#35bf28}+4.66\%$
test_sync	96.2679ms	95.8852ms	10.4291 Ops/s	9.8711 Ops/s	$\textbf{\color{#35bf28}+5.65\%}$
test_async	0.1811s	91.7274ms	10.9019 Ops/s	10.9403 Ops/s	$\color{#d91a1a}-0.35\%$
test_single_pixels	0.1284s	0.1256s	7.9627 Ops/s	7.5983 Ops/s	$\color{#35bf28}+4.80\%$
test_sync_pixels	82.3146ms	79.8213ms	12.5280 Ops/s	12.5398 Ops/s	$\color{#d91a1a}-0.09\%$
test_async_pixels	0.1465s	75.7437ms	13.2024 Ops/s	15.1284 Ops/s	$\textbf{\color{#d91a1a}-12.73\%}$
test_simple	0.8705s	0.8152s	1.2268 Ops/s	1.2203 Ops/s	$\color{#35bf28}+0.53\%$
test_transformed	1.0858s	1.0310s	0.9699 Ops/s	0.9616 Ops/s	$\color{#35bf28}+0.86\%$
test_serial	2.4415s	2.3651s	0.4228 Ops/s	0.4151 Ops/s	$\color{#35bf28}+1.87\%$
test_parallel	2.1133s	2.0426s	0.4896 Ops/s	0.4804 Ops/s	$\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-True-True-True-True]	85.1010μs	32.1832μs	31.0721 KOps/s	30.8976 KOps/s	$\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-True-True-True-False]	45.5310μs	19.3738μs	51.6161 KOps/s	50.6105 KOps/s	$\color{#35bf28}+1.99\%$
test_step_mdp_speed[True-True-True-False-True]	36.1200μs	18.3873μs	54.3854 KOps/s	54.2174 KOps/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-True-True-False-False]	25.4610μs	10.9417μs	91.3939 KOps/s	91.0212 KOps/s	$\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-True-False-True-True]	58.5910μs	34.1669μs	29.2681 KOps/s	29.2850 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-True-False-True-False]	37.8500μs	21.0342μs	47.5417 KOps/s	46.4492 KOps/s	$\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-True-False-False-True]	46.2710μs	20.1815μs	49.5502 KOps/s	49.2318 KOps/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-True-False-False-False]	31.3110μs	12.8495μs	77.8238 KOps/s	76.7705 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-False-True-True-True]	51.5420μs	35.3416μs	28.2953 KOps/s	27.2559 KOps/s	$\color{#35bf28}+3.81\%$
test_step_mdp_speed[True-False-True-True-False]	40.2210μs	23.0744μs	43.3381 KOps/s	42.1156 KOps/s	$\color{#35bf28}+2.90\%$
test_step_mdp_speed[True-False-True-False-True]	38.1210μs	19.9395μs	50.1518 KOps/s	49.2703 KOps/s	$\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-False-True-False-False]	29.8910μs	12.7722μs	78.2949 KOps/s	77.7962 KOps/s	$\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-False-True-True]	61.9320μs	37.2097μs	26.8747 KOps/s	26.6081 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-False-False-True-False]	38.3810μs	24.8217μs	40.2873 KOps/s	39.4824 KOps/s	$\color{#35bf28}+2.04\%$
test_step_mdp_speed[True-False-False-False-True]	38.2100μs	21.6605μs	46.1669 KOps/s	45.8185 KOps/s	$\color{#35bf28}+0.76\%$
test_step_mdp_speed[True-False-False-False-False]	30.1510μs	14.6545μs	68.2384 KOps/s	67.9455 KOps/s	$\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-True-True-True-True]	57.1000μs	35.8004μs	27.9326 KOps/s	27.4272 KOps/s	$\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-True-True-True-False]	47.9700μs	23.3006μs	42.9173 KOps/s	42.6310 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-True-True-False-True]	65.2710μs	23.8417μs	41.9433 KOps/s	41.4743 KOps/s	$\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-True-True-False-False]	33.1300μs	14.6454μs	68.2809 KOps/s	67.4402 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-False-True-True]	61.3510μs	37.5838μs	26.6072 KOps/s	25.9304 KOps/s	$\color{#35bf28}+2.61\%$
test_step_mdp_speed[False-True-False-True-False]	60.5410μs	24.9162μs	40.1345 KOps/s	39.3646 KOps/s	$\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-True-False-False-True]	47.3300μs	25.5686μs	39.1105 KOps/s	38.5804 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-True-False-False-False]	33.5400μs	16.2827μs	61.4148 KOps/s	59.7513 KOps/s	$\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-False-True-True-True]	95.1010μs	39.3611μs	25.4058 KOps/s	25.0564 KOps/s	$\color{#35bf28}+1.39\%$
test_step_mdp_speed[False-False-True-True-False]	44.0400μs	26.6477μs	37.5267 KOps/s	36.7510 KOps/s	$\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-False-True-False-True]	47.7010μs	25.5940μs	39.0717 KOps/s	38.0399 KOps/s	$\color{#35bf28}+2.71\%$
test_step_mdp_speed[False-False-True-False-False]	31.6200μs	16.2913μs	61.3824 KOps/s	59.2641 KOps/s	$\color{#35bf28}+3.57\%$
test_step_mdp_speed[False-False-False-True-True]	62.4500μs	40.8946μs	24.4531 KOps/s	24.1227 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-False-False-True-False]	52.6700μs	28.6453μs	34.9097 KOps/s	34.5812 KOps/s	$\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-False-False-False-True]	47.7110μs	27.1994μs	36.7656 KOps/s	36.3654 KOps/s	$\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-False-False-False]	32.7510μs	18.0438μs	55.4206 KOps/s	54.1809 KOps/s	$\color{#35bf28}+2.29\%$
test_values[generalized_advantage_estimate-True-True]	24.9032ms	24.3059ms	41.1423 Ops/s	41.9568 Ops/s	$\color{#d91a1a}-1.94\%$
test_values[vec_generalized_advantage_estimate-True-True]	88.7009ms	3.3302ms	300.2823 Ops/s	311.2473 Ops/s	$\color{#d91a1a}-3.52\%$
test_values[td0_return_estimate-False-False]	0.1008ms	60.1834μs	16.6159 KOps/s	17.0022 KOps/s	$\color{#d91a1a}-2.27\%$
test_values[td1_return_estimate-False-False]	52.6573ms	52.1798ms	19.1645 Ops/s	19.3922 Ops/s	$\color{#d91a1a}-1.17\%$
test_values[vec_td1_return_estimate-False-False]	2.0396ms	1.7485ms	571.9131 Ops/s	573.4616 Ops/s	$\color{#d91a1a}-0.27\%$
test_values[td_lambda_return_estimate-True-False]	88.8242ms	86.4229ms	11.5710 Ops/s	12.1318 Ops/s	$\color{#d91a1a}-4.62\%$
test_values[vec_td_lambda_return_estimate-True-False]	3.9929ms	1.7876ms	559.4003 Ops/s	538.7746 Ops/s	$\color{#35bf28}+3.83\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	23.1162ms	22.9382ms	43.5955 Ops/s	43.7415 Ops/s	$\color{#d91a1a}-0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8964ms	0.6925ms	1.4441 KOps/s	1.4675 KOps/s	$\color{#d91a1a}-1.59\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7031ms	0.6412ms	1.5597 KOps/s	1.5554 KOps/s	$\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4946ms	1.4435ms	692.7512 Ops/s	696.4873 Ops/s	$\color{#d91a1a}-0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9208ms	0.6610ms	1.5129 KOps/s	1.5252 KOps/s	$\color{#d91a1a}-0.80\%$
test_dqn_speed	9.1529ms	1.4503ms	689.5290 Ops/s	683.1285 Ops/s	$\color{#35bf28}+0.94\%$
test_ddpg_speed	2.9247ms	2.7661ms	361.5188 Ops/s	353.1284 Ops/s	$\color{#35bf28}+2.38\%$
test_sac_speed	8.4314ms	7.9843ms	125.2462 Ops/s	123.3537 Ops/s	$\color{#35bf28}+1.53\%$
test_redq_speed	16.2110ms	10.0839ms	99.1682 Ops/s	98.4559 Ops/s	$\color{#35bf28}+0.72\%$
test_redq_deprec_speed	12.0533ms	11.2611ms	88.8014 Ops/s	89.5900 Ops/s	$\color{#d91a1a}-0.88\%$
test_td3_speed	15.9924ms	8.0810ms	123.7471 Ops/s	121.5684 Ops/s	$\color{#35bf28}+1.79\%$
test_cql_speed	25.9193ms	24.9878ms	40.0195 Ops/s	39.6864 Ops/s	$\color{#35bf28}+0.84\%$
test_a2c_speed	5.7015ms	5.4576ms	183.2301 Ops/s	181.6982 Ops/s	$\color{#35bf28}+0.84\%$
test_ppo_speed	7.0751ms	5.8238ms	171.7095 Ops/s	172.4392 Ops/s	$\color{#d91a1a}-0.42\%$
test_reinforce_speed	4.8207ms	4.5682ms	218.9029 Ops/s	221.0183 Ops/s	$\color{#d91a1a}-0.96\%$
test_iql_speed	20.0808ms	19.2486ms	51.9519 Ops/s	51.9200 Ops/s	$\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0055ms	2.8494ms	350.9484 Ops/s	353.8885 Ops/s	$\color{#d91a1a}-0.83\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	90.1956ms	0.5947ms	1.6816 KOps/s	1.8526 KOps/s	$\textbf{\color{#d91a1a}-9.23\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6830ms	0.5061ms	1.9759 KOps/s	1.9412 KOps/s	$\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.7799ms	2.8752ms	347.8039 Ops/s	347.6370 Ops/s	$\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2390ms	0.5286ms	1.8918 KOps/s	1.8944 KOps/s	$\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6528ms	0.5033ms	1.9871 KOps/s	1.9657 KOps/s	$\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.0947ms	2.9771ms	335.8942 Ops/s	337.0112 Ops/s	$\color{#d91a1a}-0.33\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2524ms	0.6543ms	1.5284 KOps/s	1.5134 KOps/s	$\color{#35bf28}+0.99\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8152ms	0.6307ms	1.5854 KOps/s	1.5787 KOps/s	$\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1762ms	2.8759ms	347.7116 Ops/s	352.8312 Ops/s	$\color{#d91a1a}-1.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1322ms	0.5310ms	1.8834 KOps/s	1.8563 KOps/s	$\color{#35bf28}+1.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6757ms	0.5117ms	1.9544 KOps/s	1.9492 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.2070ms	2.8861ms	346.4857 Ops/s	347.0278 Ops/s	$\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1900ms	0.5239ms	1.9089 KOps/s	1.8789 KOps/s	$\color{#35bf28}+1.60\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6621ms	0.5044ms	1.9827 KOps/s	1.9740 KOps/s	$\color{#35bf28}+0.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.1527ms	3.0066ms	332.5978 Ops/s	337.5563 Ops/s	$\color{#d91a1a}-1.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2473ms	0.6538ms	1.5296 KOps/s	1.5224 KOps/s	$\color{#35bf28}+0.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8094ms	0.6294ms	1.5888 KOps/s	1.5729 KOps/s	$\color{#35bf28}+1.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1217s	8.8635ms	112.8219 Ops/s	111.7117 Ops/s	$\color{#35bf28}+0.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	16.6551ms	14.4505ms	69.2018 Ops/s	68.3973 Ops/s	$\color{#35bf28}+1.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2594ms	1.1371ms	879.4621 Ops/s	973.5339 Ops/s	$\textbf{\color{#d91a1a}-9.66\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	99.4706ms	6.5600ms	152.4382 Ops/s	116.5588 Ops/s	$\textbf{\color{#35bf28}+30.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1109s	16.3148ms	61.2940 Ops/s	68.5068 Ops/s	$\textbf{\color{#d91a1a}-10.53\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.9803ms	1.2370ms	808.4384 Ops/s	957.6401 Ops/s	$\textbf{\color{#d91a1a}-15.58\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	99.8972ms	6.9111ms	144.6951 Ops/s	142.5526 Ops/s	$\color{#35bf28}+1.50\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.0083ms	14.5860ms	68.5587 Ops/s	67.2122 Ops/s	$\color{#35bf28}+2.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.0362ms	1.5796ms	633.0633 Ops/s	693.6797 Ops/s	$\textbf{\color{#d91a1a}-8.74\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	63.1589ms	61.3664ms	16.2956 Ops/s	16.4029 Ops/s	$\color{#d91a1a}-0.65\%$
test_sync	41.4672ms	33.4653ms	29.8817 Ops/s	30.1738 Ops/s	$\color{#d91a1a}-0.97\%$
test_async	55.1513ms	30.6374ms	32.6399 Ops/s	30.4135 Ops/s	$\textbf{\color{#35bf28}+7.32\%}$
test_simple	0.4836s	0.4309s	2.3209 Ops/s	2.2714 Ops/s	$\color{#35bf28}+2.18\%$
test_transformed	0.6402s	0.5931s	1.6859 Ops/s	1.7330 Ops/s	$\color{#d91a1a}-2.71\%$
test_serial	1.4783s	1.4330s	0.6978 Ops/s	0.7021 Ops/s	$\color{#d91a1a}-0.60\%$
test_parallel	1.4362s	1.3818s	0.7237 Ops/s	0.7194 Ops/s	$\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-True-True-True-True]	94.0340μs	20.9527μs	47.7265 KOps/s	46.9186 KOps/s	$\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-True-True-True-False]	68.9080μs	12.8629μs	77.7429 KOps/s	76.5983 KOps/s	$\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-True-True-False-True]	33.5230μs	12.3711μs	80.8332 KOps/s	78.3253 KOps/s	$\color{#35bf28}+3.20\%$
test_step_mdp_speed[True-True-True-False-False]	49.3520μs	7.6223μs	131.1936 KOps/s	131.1641 KOps/s	$\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-True-False-True-True]	46.7670μs	22.6224μs	44.2041 KOps/s	43.6781 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-False-True-False]	58.4590μs	14.3633μs	69.6219 KOps/s	69.3849 KOps/s	$\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-False-False-True]	39.2630μs	13.6526μs	73.2462 KOps/s	71.9670 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-True-False-False-False]	48.7110μs	8.8794μs	112.6196 KOps/s	112.2022 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-True-True-True]	55.5530μs	23.7831μs	42.0467 KOps/s	41.4509 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-False-True-True-False]	59.3510μs	15.5199μs	64.4334 KOps/s	64.3306 KOps/s	$\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-False-True-False-True]	48.1300μs	13.7948μs	72.4909 KOps/s	72.4339 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-False-True-False-False]	25.1670μs	8.9153μs	112.1665 KOps/s	113.2778 KOps/s	$\color{#d91a1a}-0.98\%$
test_step_mdp_speed[True-False-False-True-True]	66.1150μs	25.1231μs	39.8040 KOps/s	39.3794 KOps/s	$\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-False-True-False]	37.0990μs	16.9025μs	59.1630 KOps/s	59.6043 KOps/s	$\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-False-False-True]	48.8710μs	14.8956μs	67.1338 KOps/s	66.4264 KOps/s	$\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-False-False-False]	40.0250μs	10.0495μs	99.5076 KOps/s	99.0923 KOps/s	$\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-True-True-True-True]	58.1890μs	23.8288μs	41.9660 KOps/s	41.6381 KOps/s	$\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-True-True-True-False]	38.7530μs	15.7407μs	63.5297 KOps/s	64.1773 KOps/s	$\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-True-False-True]	60.3330μs	15.9764μs	62.5925 KOps/s	62.3687 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-True-True-False-False]	33.0520μs	10.2414μs	97.6434 KOps/s	97.5859 KOps/s	$\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-False-True-True]	37.7500μs	25.6542μs	38.9800 KOps/s	39.0854 KOps/s	$\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-True-False-True-False]	61.0740μs	16.7855μs	59.5753 KOps/s	59.0387 KOps/s	$\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-False-True]	44.6230μs	17.0937μs	58.5012 KOps/s	56.8005 KOps/s	$\color{#35bf28}+2.99\%$
test_step_mdp_speed[False-True-False-False-False]	60.6330μs	11.2978μs	88.5126 KOps/s	88.7578 KOps/s	$\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-False-True-True-True]	78.0760μs	26.6640μs	37.5038 KOps/s	37.3462 KOps/s	$\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-False-True-True-False]	39.3530μs	18.4068μs	54.3277 KOps/s	54.5572 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-True-False-True]	62.9180μs	17.2616μs	57.9321 KOps/s	57.5529 KOps/s	$\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-False-True-False-False]	35.5360μs	11.6622μs	85.7468 KOps/s	88.0999 KOps/s	$\color{#d91a1a}-2.67\%$
test_step_mdp_speed[False-False-False-True-True]	74.2290μs	27.7534μs	36.0317 KOps/s	36.1205 KOps/s	$\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-False-False-True-False]	70.1000μs	19.3725μs	51.6196 KOps/s	51.5252 KOps/s	$\color{#35bf28}+0.18\%$
test_step_mdp_speed[False-False-False-False-True]	40.0940μs	18.3840μs	54.3952 KOps/s	54.0665 KOps/s	$\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-False-False-False]	49.2920μs	12.4822μs	80.1141 KOps/s	80.9948 KOps/s	$\color{#d91a1a}-1.09\%$
test_values[generalized_advantage_estimate-True-True]	14.1043ms	9.8543ms	101.4784 Ops/s	108.6688 Ops/s	$\textbf{\color{#d91a1a}-6.62\%}$
test_values[vec_generalized_advantage_estimate-True-True]	36.7597ms	33.2526ms	30.0728 Ops/s	30.0602 Ops/s	$\color{#35bf28}+0.04\%$
test_values[td0_return_estimate-False-False]	0.2376ms	0.1629ms	6.1378 KOps/s	6.0394 KOps/s	$\color{#35bf28}+1.63\%$
test_values[td1_return_estimate-False-False]	23.5266ms	23.2398ms	43.0295 Ops/s	43.7105 Ops/s	$\color{#d91a1a}-1.56\%$
test_values[vec_td1_return_estimate-False-False]	34.2375ms	33.2970ms	30.0327 Ops/s	30.1440 Ops/s	$\color{#d91a1a}-0.37\%$
test_values[td_lambda_return_estimate-True-False]	36.5352ms	33.6886ms	29.6837 Ops/s	29.8572 Ops/s	$\color{#d91a1a}-0.58\%$
test_values[vec_td_lambda_return_estimate-True-False]	35.2532ms	33.3626ms	29.9737 Ops/s	30.0177 Ops/s	$\color{#d91a1a}-0.15\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.8576ms	8.1455ms	122.7679 Ops/s	123.0755 Ops/s	$\color{#d91a1a}-0.25\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.0848ms	1.8186ms	549.8823 Ops/s	508.3708 Ops/s	$\textbf{\color{#35bf28}+8.17\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5008ms	0.3519ms	2.8420 KOps/s	2.8574 KOps/s	$\color{#d91a1a}-0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	41.0875ms	39.0250ms	25.6246 Ops/s	25.7729 Ops/s	$\color{#d91a1a}-0.58\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.6056ms	3.0017ms	333.1427 Ops/s	330.7081 Ops/s	$\color{#35bf28}+0.74\%$
test_dqn_speed	7.0037ms	1.3863ms	721.3524 Ops/s	726.0328 Ops/s	$\color{#d91a1a}-0.64\%$
test_ddpg_speed	3.4783ms	2.7515ms	363.4328 Ops/s	359.6916 Ops/s	$\color{#35bf28}+1.04\%$
test_sac_speed	72.4381ms	8.8216ms	113.3577 Ops/s	120.7546 Ops/s	$\textbf{\color{#d91a1a}-6.13\%}$
test_redq_speed	13.8995ms	12.9559ms	77.1852 Ops/s	75.7182 Ops/s	$\color{#35bf28}+1.94\%$
test_redq_deprec_speed	14.3269ms	13.2223ms	75.6299 Ops/s	77.1879 Ops/s	$\color{#d91a1a}-2.02\%$
test_td3_speed	8.5527ms	8.2794ms	120.7813 Ops/s	120.6217 Ops/s	$\color{#35bf28}+0.13\%$
test_cql_speed	38.2832ms	36.1850ms	27.6358 Ops/s	27.7110 Ops/s	$\color{#d91a1a}-0.27\%$
test_a2c_speed	8.0050ms	7.2985ms	137.0136 Ops/s	135.8930 Ops/s	$\color{#35bf28}+0.82\%$
test_ppo_speed	9.0030ms	7.6186ms	131.2585 Ops/s	131.5717 Ops/s	$\color{#d91a1a}-0.24\%$
test_reinforce_speed	7.3462ms	6.5332ms	153.0653 Ops/s	151.4956 Ops/s	$\color{#35bf28}+1.04\%$
test_iql_speed	33.5680ms	32.2657ms	30.9927 Ops/s	29.7184 Ops/s	$\color{#35bf28}+4.29\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.3157ms	2.1137ms	473.1129 Ops/s	448.0454 Ops/s	$\textbf{\color{#35bf28}+5.59\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7781ms	0.4880ms	2.0491 KOps/s	2.0149 KOps/s	$\color{#35bf28}+1.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.3463ms	0.4610ms	2.1691 KOps/s	2.1316 KOps/s	$\color{#35bf28}+1.76\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.3231ms	2.0883ms	478.8676 Ops/s	466.8266 Ops/s	$\color{#35bf28}+2.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9169ms	0.4829ms	2.0710 KOps/s	2.0775 KOps/s	$\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7144ms	0.4576ms	2.1855 KOps/s	1.8659 KOps/s	$\textbf{\color{#35bf28}+17.13\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.3237ms	2.2177ms	450.9137 Ops/s	436.9475 Ops/s	$\color{#35bf28}+3.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0260ms	0.5986ms	1.6707 KOps/s	1.6506 KOps/s	$\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9442ms	0.5709ms	1.7515 KOps/s	1.6825 KOps/s	$\color{#35bf28}+4.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.5935ms	2.1645ms	461.9920 Ops/s	429.9329 Ops/s	$\textbf{\color{#35bf28}+7.46\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.9784ms	0.4868ms	2.0544 KOps/s	1.6635 KOps/s	$\textbf{\color{#35bf28}+23.50\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7128ms	0.4643ms	2.1537 KOps/s	2.1082 KOps/s	$\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.4839ms	2.1086ms	474.2547 Ops/s	447.6644 Ops/s	$\textbf{\color{#35bf28}+5.94\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9090ms	0.4798ms	2.0842 KOps/s	2.0840 KOps/s	$+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7207ms	0.4609ms	2.1695 KOps/s	2.1591 KOps/s	$\color{#35bf28}+0.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.3514ms	2.2224ms	449.9582 Ops/s	442.5735 Ops/s	$\color{#35bf28}+1.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7799ms	0.5980ms	1.6723 KOps/s	1.6636 KOps/s	$\color{#35bf28}+0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	89.9063ms	0.6738ms	1.4842 KOps/s	1.7186 KOps/s	$\textbf{\color{#d91a1a}-13.64\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	85.1141ms	5.1976ms	192.3956 Ops/s	178.8289 Ops/s	$\textbf{\color{#35bf28}+7.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	14.7478ms	11.9000ms	84.0337 Ops/s	84.3758 Ops/s	$\color{#d91a1a}-0.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.6449ms	1.0238ms	976.7173 Ops/s	961.5835 Ops/s	$\color{#35bf28}+1.57\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	88.8945ms	6.9057ms	144.8089 Ops/s	141.1463 Ops/s	$\color{#35bf28}+2.59\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	14.7689ms	11.8839ms	84.1471 Ops/s	84.4190 Ops/s	$\color{#d91a1a}-0.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.7180ms	1.0266ms	974.0614 Ops/s	967.6804 Ops/s	$\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	88.7099ms	7.2545ms	137.8447 Ops/s	177.1252 Ops/s	$\textbf{\color{#d91a1a}-22.18\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	14.7632ms	12.2028ms	81.9487 Ops/s	81.7594 Ops/s	$\color{#35bf28}+0.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.0415ms	1.3485ms	741.5910 Ops/s	740.0406 Ops/s	$\color{#35bf28}+0.21\%$