[Feature] ActionDiscretizer

[vmoens]

A transform to discretize continuous actions

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2247

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unrelated Failures

As of commit 7d1f59a with merge base 559b729 ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t f4d8c8b7178988ddb84c883ffc038f4c30c8e8ccf8b66cb3463a3bba45b2aeb4 /exec failed with exit code 139
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 2136a7755baf241bee60c1437494ef049b58fe013a497892fc767193d1206406 /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (trunk failure)
  AttributeError: module 'torch' has no attribute 'uint16'
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure)
  AttributeError: module 'torch' has no attribute 'uint16'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1266s	61.5613ms	16.2440 Ops/s	18.1520 Ops/s	$\textbf{\color{#d91a1a}-10.51\%}$
test_sync	33.0505ms	31.6148ms	31.6308 Ops/s	32.6234 Ops/s	$\color{#d91a1a}-3.04\%$
test_async	50.8451ms	28.7078ms	34.8338 Ops/s	32.4775 Ops/s	$\textbf{\color{#35bf28}+7.25\%}$
test_simple	0.4665s	0.4004s	2.4978 Ops/s	2.6644 Ops/s	$\textbf{\color{#d91a1a}-6.25\%}$
test_transformed	0.5499s	0.5480s	1.8247 Ops/s	1.8962 Ops/s	$\color{#d91a1a}-3.77\%$
test_serial	1.3825s	1.3043s	0.7667 Ops/s	0.7889 Ops/s	$\color{#d91a1a}-2.82\%$
test_parallel	1.1664s	1.0854s	0.9213 Ops/s	0.9084 Ops/s	$\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-True-True-True-True]	0.1886ms	22.9394μs	43.5931 KOps/s	44.7550 KOps/s	$\color{#d91a1a}-2.60\%$
test_step_mdp_speed[True-True-True-True-False]	55.8940μs	13.3419μs	74.9519 KOps/s	76.1123 KOps/s	$\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-True-True-False-True]	36.2080μs	13.3679μs	74.8059 KOps/s	76.1782 KOps/s	$\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-True-True-False-False]	39.1430μs	7.8282μs	127.7440 KOps/s	130.9870 KOps/s	$\color{#d91a1a}-2.48\%$
test_step_mdp_speed[True-True-False-True-True]	58.4290μs	24.3990μs	40.9853 KOps/s	42.3968 KOps/s	$\color{#d91a1a}-3.33\%$
test_step_mdp_speed[True-True-False-True-False]	44.4530μs	14.5925μs	68.5282 KOps/s	70.0589 KOps/s	$\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-True-False-False-True]	50.1940μs	14.7301μs	67.8882 KOps/s	70.2517 KOps/s	$\color{#d91a1a}-3.36\%$
test_step_mdp_speed[True-True-False-False-False]	0.1362ms	9.1220μs	109.6254 KOps/s	113.4260 KOps/s	$\color{#d91a1a}-3.35\%$
test_step_mdp_speed[True-False-True-True-True]	0.1001ms	25.8641μs	38.6637 KOps/s	40.1080 KOps/s	$\color{#d91a1a}-3.60\%$
test_step_mdp_speed[True-False-True-True-False]	43.1810μs	16.0250μs	62.4026 KOps/s	63.9817 KOps/s	$\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-False-True-False-True]	41.2470μs	14.6265μs	68.3690 KOps/s	69.4321 KOps/s	$\color{#d91a1a}-1.53\%$
test_step_mdp_speed[True-False-True-False-False]	45.5360μs	9.1466μs	109.3308 KOps/s	112.1924 KOps/s	$\color{#d91a1a}-2.55\%$
test_step_mdp_speed[True-False-False-True-True]	71.3730μs	26.8687μs	37.2180 KOps/s	38.0611 KOps/s	$\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-False-False-True-False]	50.7660μs	17.2930μs	57.8267 KOps/s	59.0806 KOps/s	$\color{#d91a1a}-2.12\%$
test_step_mdp_speed[True-False-False-False-True]	60.0520μs	15.7952μs	63.3103 KOps/s	63.7552 KOps/s	$\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-False-False-False-False]	52.2870μs	10.4120μs	96.0427 KOps/s	98.4872 KOps/s	$\color{#d91a1a}-2.48\%$
test_step_mdp_speed[False-True-True-True-True]	63.7590μs	25.7057μs	38.9018 KOps/s	40.0203 KOps/s	$\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-True-True-True-False]	51.4760μs	15.9731μs	62.6054 KOps/s	64.3892 KOps/s	$\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-True-True-False-True]	66.8750μs	17.1457μs	58.3236 KOps/s	59.5527 KOps/s	$\color{#d91a1a}-2.06\%$
test_step_mdp_speed[False-True-True-False-False]	51.8070μs	10.3492μs	96.6255 KOps/s	98.3898 KOps/s	$\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-True-False-True-True]	57.2670μs	27.1422μs	36.8431 KOps/s	37.9438 KOps/s	$\color{#d91a1a}-2.90\%$
test_step_mdp_speed[False-True-False-True-False]	54.4010μs	17.2947μs	57.8211 KOps/s	60.2533 KOps/s	$\color{#d91a1a}-4.04\%$
test_step_mdp_speed[False-True-False-False-True]	44.1030μs	18.6385μs	53.6523 KOps/s	55.4921 KOps/s	$\color{#d91a1a}-3.32\%$
test_step_mdp_speed[False-True-False-False-False]	43.8120μs	11.6180μs	86.0732 KOps/s	88.4028 KOps/s	$\color{#d91a1a}-2.64\%$
test_step_mdp_speed[False-False-True-True-True]	68.3080μs	28.9064μs	34.5944 KOps/s	36.0731 KOps/s	$\color{#d91a1a}-4.10\%$
test_step_mdp_speed[False-False-True-True-False]	54.4430μs	18.8058μs	53.1750 KOps/s	55.5079 KOps/s	$\color{#d91a1a}-4.20\%$
test_step_mdp_speed[False-False-True-False-True]	54.4220μs	18.4403μs	54.2289 KOps/s	55.8974 KOps/s	$\color{#d91a1a}-2.98\%$
test_step_mdp_speed[False-False-True-False-False]	33.8730μs	11.6314μs	85.9739 KOps/s	89.4122 KOps/s	$\color{#d91a1a}-3.85\%$
test_step_mdp_speed[False-False-False-True-True]	62.3860μs	30.1412μs	33.1772 KOps/s	33.7774 KOps/s	$\color{#d91a1a}-1.78\%$
test_step_mdp_speed[False-False-False-True-False]	51.5460μs	19.9137μs	50.2166 KOps/s	52.3194 KOps/s	$\color{#d91a1a}-4.02\%$
test_step_mdp_speed[False-False-False-False-True]	61.8350μs	19.4341μs	51.4559 KOps/s	52.2595 KOps/s	$\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-False-False-False-False]	54.2810μs	12.7564μs	78.3921 KOps/s	80.2525 KOps/s	$\color{#d91a1a}-2.32\%$
test_values[generalized_advantage_estimate-True-True]	10.5926ms	9.8855ms	101.1584 Ops/s	98.5011 Ops/s	$\color{#35bf28}+2.70\%$
test_values[vec_generalized_advantage_estimate-True-True]	38.8379ms	35.4999ms	28.1691 Ops/s	27.9497 Ops/s	$\color{#35bf28}+0.78\%$
test_values[td0_return_estimate-False-False]	0.2170ms	0.1725ms	5.7958 KOps/s	6.0907 KOps/s	$\color{#d91a1a}-4.84\%$
test_values[td1_return_estimate-False-False]	26.8993ms	24.4183ms	40.9530 Ops/s	39.0308 Ops/s	$\color{#35bf28}+4.92\%$
test_values[vec_td1_return_estimate-False-False]	36.7626ms	35.5567ms	28.1241 Ops/s	28.1282 Ops/s	$\color{#d91a1a}-0.01\%$
test_values[td_lambda_return_estimate-True-False]	38.0457ms	35.2097ms	28.4013 Ops/s	27.2288 Ops/s	$\color{#35bf28}+4.31\%$
test_values[vec_td_lambda_return_estimate-True-False]	37.2839ms	35.6552ms	28.0464 Ops/s	28.0422 Ops/s	$\color{#35bf28}+0.02\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.7065ms	8.5605ms	116.8162 Ops/s	113.7573 Ops/s	$\color{#35bf28}+2.69\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.9502ms	1.9527ms	512.1235 Ops/s	526.2565 Ops/s	$\color{#d91a1a}-2.69\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5803ms	0.3586ms	2.7883 KOps/s	2.7476 KOps/s	$\color{#35bf28}+1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	57.9110ms	47.6184ms	21.0003 Ops/s	23.7257 Ops/s	$\textbf{\color{#d91a1a}-11.49\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.6812ms	3.0364ms	329.3335 Ops/s	327.8175 Ops/s	$\color{#35bf28}+0.46\%$
test_dqn_speed	1.8166ms	1.3779ms	725.7167 Ops/s	747.7392 Ops/s	$\color{#d91a1a}-2.95\%$
test_ddpg_speed	3.6153ms	2.9127ms	343.3204 Ops/s	353.3411 Ops/s	$\color{#d91a1a}-2.84\%$
test_sac_speed	10.3630ms	8.6838ms	115.1566 Ops/s	115.7527 Ops/s	$\color{#d91a1a}-0.51\%$
test_redq_speed	15.4812ms	13.6360ms	73.3354 Ops/s	72.3165 Ops/s	$\color{#35bf28}+1.41\%$
test_redq_deprec_speed	14.4547ms	13.3983ms	74.6365 Ops/s	69.1275 Ops/s	$\textbf{\color{#35bf28}+7.97\%}$
test_td3_speed	17.2934ms	8.5938ms	116.3631 Ops/s	115.3169 Ops/s	$\color{#35bf28}+0.91\%$
test_cql_speed	39.4249ms	37.8786ms	26.4001 Ops/s	27.0482 Ops/s	$\color{#d91a1a}-2.40\%$
test_a2c_speed	8.4272ms	7.8171ms	127.9244 Ops/s	130.6356 Ops/s	$\color{#d91a1a}-2.08\%$
test_ppo_speed	9.5193ms	8.1085ms	123.3279 Ops/s	126.0400 Ops/s	$\color{#d91a1a}-2.15\%$
test_reinforce_speed	8.9045ms	7.0286ms	142.2753 Ops/s	144.5820 Ops/s	$\color{#d91a1a}-1.60\%$
test_iql_speed	34.9621ms	33.5940ms	29.7672 Ops/s	29.7233 Ops/s	$\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.1733ms	3.5074ms	285.1100 Ops/s	271.4297 Ops/s	$\textbf{\color{#35bf28}+5.04\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7498ms	0.4945ms	2.0224 KOps/s	2.0059 KOps/s	$\color{#35bf28}+0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.4665ms	0.4775ms	2.0942 KOps/s	2.1174 KOps/s	$\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.8897ms	3.5218ms	283.9444 Ops/s	274.6002 Ops/s	$\color{#35bf28}+3.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5924ms	0.4821ms	2.0744 KOps/s	1.9846 KOps/s	$\color{#35bf28}+4.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.5906ms	0.4634ms	2.1578 KOps/s	2.0839 KOps/s	$\color{#35bf28}+3.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2678ms	1.7205ms	581.2312 Ops/s	568.4210 Ops/s	$\color{#35bf28}+2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.6190ms	1.6448ms	607.9807 Ops/s	598.3857 Ops/s	$\color{#35bf28}+1.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.4386ms	3.6799ms	271.7476 Ops/s	255.9807 Ops/s	$\textbf{\color{#35bf28}+6.16\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0200ms	0.6336ms	1.5782 KOps/s	1.3411 KOps/s	$\textbf{\color{#35bf28}+17.68\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.7930ms	0.6088ms	1.6426 KOps/s	1.6381 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.3938ms	3.5486ms	281.8046 Ops/s	271.6443 Ops/s	$\color{#35bf28}+3.74\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7335ms	0.4951ms	2.0198 KOps/s	1.9961 KOps/s	$\color{#35bf28}+1.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.5638ms	0.4739ms	2.1101 KOps/s	2.0811 KOps/s	$\color{#35bf28}+1.39\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.5892ms	3.5333ms	283.0244 Ops/s	276.5984 Ops/s	$\color{#35bf28}+2.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3302ms	0.4966ms	2.0136 KOps/s	2.0451 KOps/s	$\color{#d91a1a}-1.54\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7464ms	0.4820ms	2.0748 KOps/s	2.0891 KOps/s	$\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.6607ms	3.7029ms	270.0577 Ops/s	262.6546 Ops/s	$\color{#35bf28}+2.82\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0700ms	0.6320ms	1.5824 KOps/s	1.5224 KOps/s	$\color{#35bf28}+3.94\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.7117ms	0.6119ms	1.6343 KOps/s	1.6424 KOps/s	$\color{#d91a1a}-0.49\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1238s	6.2220ms	160.7192 Ops/s	157.0405 Ops/s	$\color{#35bf28}+2.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1175s	14.7284ms	67.8962 Ops/s	65.6634 Ops/s	$\color{#35bf28}+3.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.7369ms	1.0657ms	938.3737 Ops/s	943.3332 Ops/s	$\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1121s	6.0360ms	165.6731 Ops/s	169.3542 Ops/s	$\color{#d91a1a}-2.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.3019ms	12.6191ms	79.2451 Ops/s	78.1249 Ops/s	$\color{#35bf28}+1.43\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.7682ms	1.0762ms	929.2217 Ops/s	953.5917 Ops/s	$\color{#d91a1a}-2.56\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1096s	6.1517ms	162.5555 Ops/s	163.8100 Ops/s	$\color{#d91a1a}-0.77\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	16.1603ms	12.8612ms	77.7531 Ops/s	76.8876 Ops/s	$\color{#35bf28}+1.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.7787ms	1.2272ms	814.8538 Ops/s	836.4138 Ops/s	$\color{#d91a1a}-2.58\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1758s	0.1240s	8.0677 Ops/s	8.4137 Ops/s	$\color{#d91a1a}-4.11\%$
test_sync	0.1040s	0.1015s	9.8534 Ops/s	10.0329 Ops/s	$\color{#d91a1a}-1.79\%$
test_async	0.1974s	98.1861ms	10.1847 Ops/s	10.7519 Ops/s	$\textbf{\color{#d91a1a}-5.28\%}$
test_single_pixels	0.1280s	0.1276s	7.8353 Ops/s	7.7460 Ops/s	$\color{#35bf28}+1.15\%$
test_sync_pixels	85.4459ms	82.6895ms	12.0934 Ops/s	12.2055 Ops/s	$\color{#d91a1a}-0.92\%$
test_async_pixels	0.1657s	69.9887ms	14.2880 Ops/s	14.2790 Ops/s	$\color{#35bf28}+0.06\%$
test_simple	0.8799s	0.8141s	1.2284 Ops/s	1.2138 Ops/s	$\color{#35bf28}+1.20\%$
test_transformed	1.1365s	1.0714s	0.9334 Ops/s	0.9274 Ops/s	$\color{#35bf28}+0.64\%$
test_serial	2.5323s	2.4706s	0.4048 Ops/s	0.3945 Ops/s	$\color{#35bf28}+2.60\%$
test_parallel	2.4252s	2.3765s	0.4208 Ops/s	0.4210 Ops/s	$\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-True-True-True]	82.1510μs	33.7607μs	29.6202 KOps/s	29.0803 KOps/s	$\color{#35bf28}+1.86\%$
test_step_mdp_speed[True-True-True-True-False]	43.1500μs	19.3216μs	51.7555 KOps/s	49.9619 KOps/s	$\color{#35bf28}+3.59\%$
test_step_mdp_speed[True-True-True-False-True]	51.8110μs	19.1402μs	52.2461 KOps/s	51.1312 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-True-True-False-False]	31.9400μs	11.0007μs	90.9033 KOps/s	87.4927 KOps/s	$\color{#35bf28}+3.90\%$
test_step_mdp_speed[True-True-False-True-True]	76.3510μs	35.6943μs	28.0157 KOps/s	27.7596 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-True-False]	49.1400μs	21.4137μs	46.6991 KOps/s	46.2948 KOps/s	$\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-True-False-False-True]	43.1410μs	20.6093μs	48.5218 KOps/s	46.9056 KOps/s	$\color{#35bf28}+3.45\%$
test_step_mdp_speed[True-True-False-False-False]	35.4200μs	12.9347μs	77.3115 KOps/s	75.8126 KOps/s	$\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-False-True-True-True]	69.8310μs	37.4825μs	26.6791 KOps/s	26.2402 KOps/s	$\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-True-True-False]	45.0410μs	23.0117μs	43.4562 KOps/s	42.5360 KOps/s	$\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-True-False-True]	44.1910μs	20.5625μs	48.6323 KOps/s	47.1819 KOps/s	$\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-False-True-False-False]	30.4710μs	12.6717μs	78.9160 KOps/s	75.8660 KOps/s	$\color{#35bf28}+4.02\%$
test_step_mdp_speed[True-False-False-True-True]	72.2420μs	38.4221μs	26.0267 KOps/s	24.8828 KOps/s	$\color{#35bf28}+4.60\%$
test_step_mdp_speed[True-False-False-True-False]	52.6110μs	24.4690μs	40.8681 KOps/s	39.5797 KOps/s	$\color{#35bf28}+3.26\%$
test_step_mdp_speed[True-False-False-False-True]	40.9400μs	22.4086μs	44.6256 KOps/s	43.8990 KOps/s	$\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-False-False-False]	31.9610μs	14.5058μs	68.9377 KOps/s	66.5261 KOps/s	$\color{#35bf28}+3.62\%$
test_step_mdp_speed[False-True-True-True-True]	73.0520μs	36.5873μs	27.3319 KOps/s	26.1287 KOps/s	$\color{#35bf28}+4.61\%$
test_step_mdp_speed[False-True-True-True-False]	47.8310μs	22.6848μs	44.0823 KOps/s	42.2950 KOps/s	$\color{#35bf28}+4.23\%$
test_step_mdp_speed[False-True-True-False-True]	47.2510μs	24.0451μs	41.5885 KOps/s	38.9899 KOps/s	$\textbf{\color{#35bf28}+6.67\%}$
test_step_mdp_speed[False-True-True-False-False]	38.3700μs	14.5006μs	68.9626 KOps/s	67.0282 KOps/s	$\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-True-False-True-True]	62.1510μs	38.3930μs	26.0464 KOps/s	25.1094 KOps/s	$\color{#35bf28}+3.73\%$
test_step_mdp_speed[False-True-False-True-False]	45.7010μs	24.5306μs	40.7655 KOps/s	39.5725 KOps/s	$\color{#35bf28}+3.01\%$
test_step_mdp_speed[False-True-False-False-True]	46.6510μs	25.7191μs	38.8816 KOps/s	36.6559 KOps/s	$\textbf{\color{#35bf28}+6.07\%}$
test_step_mdp_speed[False-True-False-False-False]	35.3100μs	16.1091μs	62.0767 KOps/s	59.0599 KOps/s	$\textbf{\color{#35bf28}+5.11\%}$
test_step_mdp_speed[False-False-True-True-True]	0.1047ms	40.2335μs	24.8549 KOps/s	23.8178 KOps/s	$\color{#35bf28}+4.35\%$
test_step_mdp_speed[False-False-True-True-False]	49.7510μs	26.5165μs	37.7124 KOps/s	36.6622 KOps/s	$\color{#35bf28}+2.86\%$
test_step_mdp_speed[False-False-True-False-True]	44.4110μs	25.8604μs	38.6691 KOps/s	36.7559 KOps/s	$\textbf{\color{#35bf28}+5.21\%}$
test_step_mdp_speed[False-False-True-False-False]	33.5410μs	16.2721μs	61.4547 KOps/s	59.9336 KOps/s	$\color{#35bf28}+2.54\%$
test_step_mdp_speed[False-False-False-True-True]	57.4210μs	43.6468μs	22.9112 KOps/s	22.4892 KOps/s	$\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-False-True-False]	54.5310μs	28.8318μs	34.6840 KOps/s	33.9260 KOps/s	$\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-False-False-False-True]	47.4800μs	27.9499μs	35.7783 KOps/s	34.3878 KOps/s	$\color{#35bf28}+4.04\%$
test_step_mdp_speed[False-False-False-False-False]	43.3100μs	18.0696μs	55.3415 KOps/s	53.7972 KOps/s	$\color{#35bf28}+2.87\%$
test_values[generalized_advantage_estimate-True-True]	25.4239ms	24.8336ms	40.2680 Ops/s	39.0341 Ops/s	$\color{#35bf28}+3.16\%$
test_values[vec_generalized_advantage_estimate-True-True]	98.1473ms	2.8686ms	348.6058 Ops/s	364.6287 Ops/s	$\color{#d91a1a}-4.39\%$
test_values[td0_return_estimate-False-False]	88.5520μs	64.9586μs	15.3944 KOps/s	14.7226 KOps/s	$\color{#35bf28}+4.56\%$
test_values[td1_return_estimate-False-False]	55.5598ms	55.2061ms	18.1140 Ops/s	17.3897 Ops/s	$\color{#35bf28}+4.16\%$
test_values[vec_td1_return_estimate-False-False]	1.4085ms	1.0905ms	917.0038 Ops/s	903.5277 Ops/s	$\color{#35bf28}+1.49\%$
test_values[td_lambda_return_estimate-True-False]	88.1784ms	87.6707ms	11.4063 Ops/s	11.0202 Ops/s	$\color{#35bf28}+3.50\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.2832ms	1.0849ms	921.7181 Ops/s	908.7841 Ops/s	$\color{#35bf28}+1.42\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.8566ms	25.1157ms	39.8157 Ops/s	38.4376 Ops/s	$\color{#35bf28}+3.59\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9407ms	0.7236ms	1.3820 KOps/s	1.3413 KOps/s	$\color{#35bf28}+3.04\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7749ms	0.6739ms	1.4839 KOps/s	1.4498 KOps/s	$\color{#35bf28}+2.36\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5725ms	1.4733ms	678.7479 Ops/s	670.1310 Ops/s	$\color{#35bf28}+1.29\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7510ms	0.6916ms	1.4458 KOps/s	1.4144 KOps/s	$\color{#35bf28}+2.22\%$
test_dqn_speed	1.8633ms	1.4462ms	691.4726 Ops/s	687.7041 Ops/s	$\color{#35bf28}+0.55\%$
test_ddpg_speed	3.3261ms	2.9575ms	338.1189 Ops/s	340.9231 Ops/s	$\color{#d91a1a}-0.82\%$
test_sac_speed	8.7033ms	8.4601ms	118.2018 Ops/s	117.9184 Ops/s	$\color{#35bf28}+0.24\%$
test_redq_speed	11.9840ms	10.7348ms	93.1553 Ops/s	93.7516 Ops/s	$\color{#d91a1a}-0.64\%$
test_redq_deprec_speed	0.1074s	12.8048ms	78.0956 Ops/s	84.9666 Ops/s	$\textbf{\color{#d91a1a}-8.09\%}$
test_td3_speed	8.6319ms	8.4592ms	118.2148 Ops/s	117.5651 Ops/s	$\color{#35bf28}+0.55\%$
test_cql_speed	27.5997ms	26.0775ms	38.3472 Ops/s	39.1078 Ops/s	$\color{#d91a1a}-1.94\%$
test_a2c_speed	6.3574ms	5.7441ms	174.0926 Ops/s	183.5519 Ops/s	$\textbf{\color{#d91a1a}-5.15\%}$
test_ppo_speed	6.2087ms	6.0504ms	165.2796 Ops/s	172.4185 Ops/s	$\color{#d91a1a}-4.14\%$
test_reinforce_speed	4.8948ms	4.7069ms	212.4540 Ops/s	225.3499 Ops/s	$\textbf{\color{#d91a1a}-5.72\%}$
test_iql_speed	20.2239ms	19.6421ms	50.9111 Ops/s	52.3704 Ops/s	$\color{#d91a1a}-2.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.7957ms	4.6079ms	217.0198 Ops/s	213.7629 Ops/s	$\color{#35bf28}+1.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0897ms	0.5929ms	1.6865 KOps/s	1.6859 KOps/s	$\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7279ms	0.5693ms	1.7566 KOps/s	1.7561 KOps/s	$\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.8833ms	4.5900ms	217.8665 Ops/s	215.9378 Ops/s	$\color{#35bf28}+0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6915ms	0.5832ms	1.7148 KOps/s	1.6976 KOps/s	$\color{#35bf28}+1.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.4577ms	0.5686ms	1.7587 KOps/s	1.7737 KOps/s	$\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2282ms	2.0983ms	476.5795 Ops/s	464.5635 Ops/s	$\color{#35bf28}+2.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	5.7826ms	2.0091ms	497.7327 Ops/s	489.2582 Ops/s	$\color{#35bf28}+1.73\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.9380ms	4.7237ms	211.6974 Ops/s	209.2221 Ops/s	$\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8815ms	0.7404ms	1.3506 KOps/s	1.3451 KOps/s	$\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.3652ms	0.7210ms	1.3869 KOps/s	1.3916 KOps/s	$\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.7686ms	4.6201ms	216.4458 Ops/s	214.5844 Ops/s	$\color{#35bf28}+0.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.3837ms	0.5940ms	1.6835 KOps/s	1.6824 KOps/s	$\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7850ms	0.5683ms	1.7597 KOps/s	1.7318 KOps/s	$\color{#35bf28}+1.61\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.8303ms	4.5680ms	218.9148 Ops/s	216.3615 Ops/s	$\color{#35bf28}+1.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6840ms	0.5878ms	1.7013 KOps/s	1.6890 KOps/s	$\color{#35bf28}+0.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.5098ms	0.5699ms	1.7548 KOps/s	1.7714 KOps/s	$\color{#d91a1a}-0.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.9621ms	4.7961ms	208.5042 Ops/s	206.5652 Ops/s	$\color{#35bf28}+0.94\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.5008ms	0.7449ms	1.3425 KOps/s	1.3447 KOps/s	$\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8852ms	0.7121ms	1.4043 KOps/s	1.3749 KOps/s	$\color{#35bf28}+2.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1326s	7.4774ms	133.7355 Ops/s	134.7580 Ops/s	$\color{#d91a1a}-0.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.9905ms	15.6521ms	63.8891 Ops/s	61.6285 Ops/s	$\color{#35bf28}+3.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.3296ms	1.2544ms	797.1904 Ops/s	771.8241 Ops/s	$\color{#35bf28}+3.29\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1238s	7.3600ms	135.8695 Ops/s	100.1389 Ops/s	$\textbf{\color{#35bf28}+35.68\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1353s	18.0997ms	55.2495 Ops/s	61.3321 Ops/s	$\textbf{\color{#d91a1a}-9.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.5800ms	1.3654ms	732.3675 Ops/s	775.5420 Ops/s	$\textbf{\color{#d91a1a}-5.57\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1236s	7.4911ms	133.4922 Ops/s	130.8427 Ops/s	$\color{#35bf28}+2.02\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.2513ms	15.8402ms	63.1305 Ops/s	60.6559 Ops/s	$\color{#35bf28}+4.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.1780ms	1.5630ms	639.7798 Ops/s	682.0215 Ops/s	$\textbf{\color{#d91a1a}-6.19\%}$