[Feature] Atari DQN dataset

[vmoens]

Docstrings can be read here

cc @agarwl: do you think we could add this in https://github.com/mila-iqia/SGI/blob/master/src/offline_dataset.py after the release (next week)?

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1815

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 1 Unrelated Failure

As of commit 0e0982d with merge base a10cdbf ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  test/test_libs.py::TestHabitat::test_habitat_render[False-HabitatPick-v0]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1171s	0.1163s	8.6011 Ops/s	8.6407 Ops/s	$\color{#d91a1a}-0.46\%$
test_sync	0.1717s	0.1026s	9.7429 Ops/s	9.7170 Ops/s	$\color{#35bf28}+0.27\%$
test_async	0.1819s	91.9587ms	10.8744 Ops/s	10.8712 Ops/s	$\color{#35bf28}+0.03\%$
test_single_pixels	0.1292s	0.1271s	7.8649 Ops/s	7.1455 Ops/s	$\textbf{\color{#35bf28}+10.07\%}$
test_sync_pixels	79.0428ms	76.6967ms	13.0384 Ops/s	12.6919 Ops/s	$\color{#35bf28}+2.73\%$
test_async_pixels	0.1374s	72.5443ms	13.7847 Ops/s	13.8686 Ops/s	$\color{#d91a1a}-0.61\%$
test_simple	0.9140s	0.8439s	1.1850 Ops/s	1.2036 Ops/s	$\color{#d91a1a}-1.54\%$
test_transformed	1.0640s	1.0633s	0.9405 Ops/s	0.9349 Ops/s	$\color{#35bf28}+0.59\%$
test_serial	2.3872s	2.3229s	0.4305 Ops/s	0.4281 Ops/s	$\color{#35bf28}+0.57\%$
test_parallel	2.0803s	1.9059s	0.5247 Ops/s	0.5348 Ops/s	$\color{#d91a1a}-1.89\%$
test_step_mdp_speed[True-True-True-True-True]	75.1410μs	33.1167μs	30.1962 KOps/s	30.5142 KOps/s	$\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-True-True-True-False]	38.5710μs	19.9424μs	50.1444 KOps/s	50.2110 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-False-True]	41.2800μs	18.5237μs	53.9850 KOps/s	53.6888 KOps/s	$\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-True-True-False-False]	36.0210μs	11.1926μs	89.3445 KOps/s	89.5992 KOps/s	$\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-False-True-True]	54.8910μs	34.9665μs	28.5988 KOps/s	29.0366 KOps/s	$\color{#d91a1a}-1.51\%$
test_step_mdp_speed[True-True-False-True-False]	39.6200μs	21.6689μs	46.1491 KOps/s	46.7448 KOps/s	$\color{#d91a1a}-1.27\%$
test_step_mdp_speed[True-True-False-False-True]	49.1610μs	20.4192μs	48.9735 KOps/s	49.3891 KOps/s	$\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-False-False-False]	32.3610μs	13.0812μs	76.4458 KOps/s	76.3410 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-True-True-True]	57.5310μs	36.8670μs	27.1245 KOps/s	27.2284 KOps/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-False-True-True-False]	45.9210μs	23.6148μs	42.3463 KOps/s	42.2890 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-True-False-True]	48.3510μs	20.2450μs	49.3949 KOps/s	48.7374 KOps/s	$\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-False-True-False-False]	32.8500μs	13.0943μs	76.3693 KOps/s	75.1811 KOps/s	$\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-False-True-True]	64.2510μs	38.8013μs	25.7723 KOps/s	26.1435 KOps/s	$\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-False-False-True-False]	48.6700μs	25.5758μs	39.0995 KOps/s	39.2905 KOps/s	$\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-False-False-False-True]	46.0310μs	22.2690μs	44.9055 KOps/s	45.1409 KOps/s	$\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-False-False-False]	32.9800μs	14.9190μs	67.0285 KOps/s	67.4081 KOps/s	$\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-True-True-True-True]	77.2920μs	37.1599μs	26.9107 KOps/s	27.1295 KOps/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-True-True-True-False]	53.5010μs	23.5118μs	42.5318 KOps/s	42.6264 KOps/s	$\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-True-True-False-True]	45.8410μs	23.9668μs	41.7243 KOps/s	41.0519 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-True-False-False]	41.4210μs	14.8866μs	67.1745 KOps/s	66.4624 KOps/s	$\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-False-True-True]	79.3410μs	38.6827μs	25.8513 KOps/s	26.2776 KOps/s	$\color{#d91a1a}-1.62\%$
test_step_mdp_speed[False-True-False-True-False]	56.6810μs	25.9702μs	38.5057 KOps/s	39.6142 KOps/s	$\color{#d91a1a}-2.80\%$
test_step_mdp_speed[False-True-False-False-True]	51.0610μs	26.2515μs	38.0931 KOps/s	39.3235 KOps/s	$\color{#d91a1a}-3.13\%$
test_step_mdp_speed[False-True-False-False-False]	57.0310μs	16.8067μs	59.5002 KOps/s	60.2476 KOps/s	$\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-False-True-True-True]	64.6510μs	40.4296μs	24.7343 KOps/s	25.1762 KOps/s	$\color{#d91a1a}-1.76\%$
test_step_mdp_speed[False-False-True-True-False]	46.5000μs	27.4158μs	36.4753 KOps/s	37.2471 KOps/s	$\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-False-True-False-True]	51.5310μs	25.5285μs	39.1719 KOps/s	38.9293 KOps/s	$\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-False-True-False-False]	35.5910μs	16.6657μs	60.0035 KOps/s	59.7161 KOps/s	$\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-False-False-True-True]	67.1010μs	42.0520μs	23.7801 KOps/s	23.7930 KOps/s	$\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-False-False-True-False]	49.7910μs	28.9818μs	34.5044 KOps/s	34.8339 KOps/s	$\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-False-False-True]	49.3510μs	27.5321μs	36.3212 KOps/s	37.0445 KOps/s	$\color{#d91a1a}-1.95\%$
test_step_mdp_speed[False-False-False-False-False]	45.6610μs	18.7604μs	53.3038 KOps/s	54.2605 KOps/s	$\color{#d91a1a}-1.76\%$
test_values[generalized_advantage_estimate-True-True]	28.2515ms	27.0823ms	36.9244 Ops/s	38.6746 Ops/s	$\color{#d91a1a}-4.53\%$
test_values[vec_generalized_advantage_estimate-True-True]	88.5038ms	3.3392ms	299.4754 Ops/s	307.7486 Ops/s	$\color{#d91a1a}-2.69\%$
test_values[td0_return_estimate-False-False]	91.0110μs	63.4353μs	15.7641 KOps/s	16.1252 KOps/s	$\color{#d91a1a}-2.24\%$
test_values[td1_return_estimate-False-False]	59.0149ms	58.0128ms	17.2376 Ops/s	18.6384 Ops/s	$\textbf{\color{#d91a1a}-7.52\%}$
test_values[vec_td1_return_estimate-False-False]	2.1249ms	1.7882ms	559.2217 Ops/s	552.6371 Ops/s	$\color{#35bf28}+1.19\%$
test_values[td_lambda_return_estimate-True-False]	95.1051ms	93.1536ms	10.7350 Ops/s	11.6355 Ops/s	$\textbf{\color{#d91a1a}-7.74\%}$
test_values[vec_td_lambda_return_estimate-True-False]	2.0601ms	1.7969ms	556.4991 Ops/s	566.0408 Ops/s	$\color{#d91a1a}-1.69\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.4344ms	24.9123ms	40.1407 Ops/s	42.5736 Ops/s	$\textbf{\color{#d91a1a}-5.71\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9339ms	0.7125ms	1.4034 KOps/s	1.3968 KOps/s	$\color{#35bf28}+0.47\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7393ms	0.6873ms	1.4550 KOps/s	1.5191 KOps/s	$\color{#d91a1a}-4.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5401ms	1.4827ms	674.4316 Ops/s	683.8876 Ops/s	$\color{#d91a1a}-1.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9631ms	0.7068ms	1.4149 KOps/s	1.4718 KOps/s	$\color{#d91a1a}-3.86\%$
test_dqn_speed	7.8688ms	7.3441ms	136.1629 Ops/s	137.1443 Ops/s	$\color{#d91a1a}-0.72\%$
test_ddpg_speed	0.1010s	15.8178ms	63.2199 Ops/s	69.5576 Ops/s	$\textbf{\color{#d91a1a}-9.11\%}$
test_sac_speed	30.0986ms	29.4181ms	33.9927 Ops/s	34.7642 Ops/s	$\color{#d91a1a}-2.22\%$
test_redq_speed	50.4758ms	48.4779ms	20.6279 Ops/s	21.1746 Ops/s	$\color{#d91a1a}-2.58\%$
test_redq_deprec_speed	25.0346ms	24.0688ms	41.5475 Ops/s	41.9742 Ops/s	$\color{#d91a1a}-1.02\%$
test_td3_speed	29.2146ms	19.5965ms	51.0295 Ops/s	50.9837 Ops/s	$\color{#35bf28}+0.09\%$
test_cql_speed	83.1340ms	81.7258ms	12.2360 Ops/s	12.1952 Ops/s	$\color{#35bf28}+0.33\%$
test_a2c_speed	27.3600ms	26.0970ms	38.3186 Ops/s	38.0223 Ops/s	$\color{#35bf28}+0.78\%$
test_ppo_speed	27.2443ms	26.4091ms	37.8657 Ops/s	37.3927 Ops/s	$\color{#35bf28}+1.27\%$
test_reinforce_speed	26.1080ms	25.1572ms	39.7501 Ops/s	39.3985 Ops/s	$\color{#35bf28}+0.89\%$
test_iql_speed	56.8452ms	56.1122ms	17.8214 Ops/s	17.6454 Ops/s	$\color{#35bf28}+1.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.2566ms	1.8694ms	534.9343 Ops/s	539.4562 Ops/s	$\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.9629ms	0.8421ms	1.1875 KOps/s	1.1843 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.9425ms	0.8184ms	1.2219 KOps/s	1.2203 KOps/s	$\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.7303ms	1.8268ms	547.4069 Ops/s	550.9535 Ops/s	$\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9822ms	0.8312ms	1.2030 KOps/s	1.2031 KOps/s	$-0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.9524ms	0.8086ms	1.2367 KOps/s	1.2354 KOps/s	$\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	0.1242s	2.3496ms	425.6040 Ops/s	482.2052 Ops/s	$\textbf{\color{#d91a1a}-11.74\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0853ms	0.9571ms	1.0448 KOps/s	1.0372 KOps/s	$\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0505ms	0.9355ms	1.0690 KOps/s	1.0637 KOps/s	$\color{#35bf28}+0.49\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.5905ms	1.8522ms	539.8947 Ops/s	543.4935 Ops/s	$\color{#d91a1a}-0.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.9631ms	0.8418ms	1.1879 KOps/s	1.1840 KOps/s	$\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.9493ms	0.8190ms	1.2210 KOps/s	1.2178 KOps/s	$\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.6086ms	1.8228ms	548.6161 Ops/s	553.4344 Ops/s	$\color{#d91a1a}-0.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1131s	0.9861ms	1.0141 KOps/s	1.1997 KOps/s	$\textbf{\color{#d91a1a}-15.48\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.9386ms	0.8074ms	1.2386 KOps/s	1.2335 KOps/s	$\color{#35bf28}+0.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.9236ms	2.0793ms	480.9298 Ops/s	483.6811 Ops/s	$\color{#d91a1a}-0.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1269ms	0.9627ms	1.0387 KOps/s	1.0402 KOps/s	$\color{#d91a1a}-0.14\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0676ms	0.9389ms	1.0651 KOps/s	1.0623 KOps/s	$\color{#35bf28}+0.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1159s	9.8583ms	101.4371 Ops/s	99.3270 Ops/s	$\color{#35bf28}+2.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	16.7491ms	14.3622ms	69.6272 Ops/s	70.5309 Ops/s	$\color{#d91a1a}-1.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.0508ms	3.4441ms	290.3530 Ops/s	288.7905 Ops/s	$\color{#35bf28}+0.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1168s	9.9153ms	100.8541 Ops/s	100.2807 Ops/s	$\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.7631ms	14.3094ms	69.8842 Ops/s	69.1079 Ops/s	$\color{#35bf28}+1.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	5.5394ms	3.4297ms	291.5711 Ops/s	287.4130 Ops/s	$\color{#35bf28}+1.45\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1201s	12.3997ms	80.6471 Ops/s	80.9013 Ops/s	$\color{#d91a1a}-0.31\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	16.5371ms	14.4739ms	69.0900 Ops/s	69.2372 Ops/s	$\color{#d91a1a}-0.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	6.0232ms	3.6278ms	275.6519 Ops/s	272.2597 Ops/s	$\color{#35bf28}+1.25\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1303s	66.0120ms	15.1488 Ops/s	16.5016 Ops/s	$\textbf{\color{#d91a1a}-8.20\%}$
test_sync	40.1081ms	34.2975ms	29.1566 Ops/s	30.6741 Ops/s	$\color{#d91a1a}-4.95\%$
test_async	61.6728ms	31.7146ms	31.5313 Ops/s	30.6917 Ops/s	$\color{#35bf28}+2.74\%$
test_simple	0.4948s	0.4370s	2.2886 Ops/s	2.3595 Ops/s	$\color{#d91a1a}-3.00\%$
test_transformed	0.6521s	0.5975s	1.6735 Ops/s	1.7255 Ops/s	$\color{#d91a1a}-3.01\%$
test_serial	1.4350s	1.3849s	0.7221 Ops/s	0.7479 Ops/s	$\color{#d91a1a}-3.45\%$
test_parallel	1.2753s	1.2069s	0.8286 Ops/s	0.8296 Ops/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-True-True]	0.1047ms	20.9746μs	47.6767 KOps/s	46.9021 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-True-True-False]	36.3480μs	12.9139μs	77.4361 KOps/s	76.3551 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-True-True-False-True]	31.8900μs	12.5326μs	79.7921 KOps/s	80.5061 KOps/s	$\color{#d91a1a}-0.89\%$
test_step_mdp_speed[True-True-True-False-False]	34.4150μs	7.5697μs	132.1055 KOps/s	130.2931 KOps/s	$\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-True-True]	75.2240μs	22.2306μs	44.9831 KOps/s	43.9782 KOps/s	$\color{#35bf28}+2.29\%$
test_step_mdp_speed[True-True-False-True-False]	41.1770μs	14.0841μs	71.0022 KOps/s	69.5247 KOps/s	$\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-False-False-True]	54.8930μs	13.7111μs	72.9334 KOps/s	73.0800 KOps/s	$\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-True-False-False-False]	36.4280μs	8.8066μs	113.5508 KOps/s	112.4581 KOps/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-True-True-True]	59.5510μs	23.6443μs	42.2935 KOps/s	41.7714 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-True-True-False]	41.6880μs	15.4183μs	64.8582 KOps/s	63.3498 KOps/s	$\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-False-True-False-True]	60.8340μs	13.6015μs	73.5211 KOps/s	73.0628 KOps/s	$\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-False-True-False-False]	29.8260μs	8.8152μs	113.4410 KOps/s	112.3460 KOps/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-False-True-True]	51.7660μs	25.0091μs	39.9854 KOps/s	39.6065 KOps/s	$\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-False-False-True-False]	41.8390μs	16.5958μs	60.2561 KOps/s	59.2113 KOps/s	$\color{#35bf28}+1.76\%$
test_step_mdp_speed[True-False-False-False-True]	70.5830μs	14.9194μs	67.0270 KOps/s	67.9320 KOps/s	$\color{#d91a1a}-1.33\%$
test_step_mdp_speed[True-False-False-False-False]	40.7260μs	10.0112μs	99.8879 KOps/s	98.8388 KOps/s	$\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-True-True-True-True]	54.6420μs	23.8328μs	41.9590 KOps/s	41.1077 KOps/s	$\color{#35bf28}+2.07\%$
test_step_mdp_speed[False-True-True-True-False]	47.8900μs	15.4180μs	64.8591 KOps/s	63.0976 KOps/s	$\color{#35bf28}+2.79\%$
test_step_mdp_speed[False-True-True-False-True]	46.0270μs	15.9760μs	62.5939 KOps/s	62.5402 KOps/s	$\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-True-True-False-False]	32.2610μs	9.8759μs	101.2564 KOps/s	98.0063 KOps/s	$\color{#35bf28}+3.32\%$
test_step_mdp_speed[False-True-False-True-True]	57.2470μs	25.1832μs	39.7089 KOps/s	39.5302 KOps/s	$\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-True-False-True-False]	45.0040μs	16.6129μs	60.1940 KOps/s	58.7225 KOps/s	$\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-True-False-False-True]	54.4320μs	17.0382μs	58.6916 KOps/s	57.7316 KOps/s	$\color{#35bf28}+1.66\%$
test_step_mdp_speed[False-True-False-False-False]	40.8560μs	11.1643μs	89.5711 KOps/s	87.7370 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-False-True-True-True]	76.1990μs	26.0245μs	38.4254 KOps/s	37.3664 KOps/s	$\color{#35bf28}+2.83\%$
test_step_mdp_speed[False-False-True-True-False]	44.2030μs	17.7019μs	56.4912 KOps/s	54.8474 KOps/s	$\color{#35bf28}+3.00\%$
test_step_mdp_speed[False-False-True-False-True]	67.2560μs	16.9294μs	59.0688 KOps/s	57.7222 KOps/s	$\color{#35bf28}+2.33\%$
test_step_mdp_speed[False-False-True-False-False]	38.0610μs	11.1351μs	89.8060 KOps/s	87.6674 KOps/s	$\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-False-False-True-True]	55.8640μs	27.1925μs	36.7749 KOps/s	36.0471 KOps/s	$\color{#35bf28}+2.02\%$
test_step_mdp_speed[False-False-False-True-False]	47.5290μs	18.8648μs	53.0089 KOps/s	52.5737 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-False-False-False-True]	49.7930μs	18.2210μs	54.8818 KOps/s	55.2736 KOps/s	$\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-False-False-False]	35.8570μs	12.1139μs	82.5495 KOps/s	81.7056 KOps/s	$\color{#35bf28}+1.03\%$
test_values[generalized_advantage_estimate-True-True]	16.4830ms	12.1095ms	82.5800 Ops/s	81.7796 Ops/s	$\color{#35bf28}+0.98\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.7674ms	28.3193ms	35.3116 Ops/s	36.6170 Ops/s	$\color{#d91a1a}-3.57\%$
test_values[td0_return_estimate-False-False]	0.2056ms	0.1820ms	5.4931 KOps/s	5.6341 KOps/s	$\color{#d91a1a}-2.50\%$
test_values[td1_return_estimate-False-False]	28.0902ms	25.0775ms	39.8764 Ops/s	38.8665 Ops/s	$\color{#35bf28}+2.60\%$
test_values[vec_td1_return_estimate-False-False]	28.4587ms	28.0601ms	35.6378 Ops/s	36.0401 Ops/s	$\color{#d91a1a}-1.12\%$
test_values[td_lambda_return_estimate-True-False]	38.5493ms	35.5211ms	28.1523 Ops/s	27.9403 Ops/s	$\color{#35bf28}+0.76\%$
test_values[vec_td_lambda_return_estimate-True-False]	0.1493s	38.1208ms	26.2324 Ops/s	36.2730 Ops/s	$\textbf{\color{#d91a1a}-27.68\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.0901ms	8.0061ms	124.9041 Ops/s	122.3874 Ops/s	$\color{#35bf28}+2.06\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4988ms	1.9514ms	512.4505 Ops/s	509.3936 Ops/s	$\color{#35bf28}+0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	8.3508ms	0.4438ms	2.2532 KOps/s	2.2839 KOps/s	$\color{#d91a1a}-1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	43.6868ms	39.2161ms	25.4997 Ops/s	24.8750 Ops/s	$\color{#35bf28}+2.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	11.2909ms	2.6585ms	376.1454 Ops/s	377.9833 Ops/s	$\color{#d91a1a}-0.49\%$
test_dqn_speed	80.1323ms	8.2362ms	121.4147 Ops/s	124.7089 Ops/s	$\color{#d91a1a}-2.64\%$
test_ddpg_speed	20.8531ms	14.8601ms	67.2941 Ops/s	69.0003 Ops/s	$\color{#d91a1a}-2.47\%$
test_sac_speed	31.1917ms	29.7558ms	33.6069 Ops/s	33.9616 Ops/s	$\color{#d91a1a}-1.04\%$
test_redq_speed	49.0064ms	45.9436ms	21.7658 Ops/s	21.9401 Ops/s	$\color{#d91a1a}-0.79\%$
test_redq_deprec_speed	35.7270ms	26.9117ms	37.1585 Ops/s	37.5907 Ops/s	$\color{#d91a1a}-1.15\%$
test_td3_speed	29.6798ms	21.0695ms	47.4619 Ops/s	48.2935 Ops/s	$\color{#d91a1a}-1.72\%$
test_cql_speed	91.4037ms	87.2607ms	11.4599 Ops/s	11.1571 Ops/s	$\color{#35bf28}+2.71\%$
test_a2c_speed	34.3708ms	26.9526ms	37.1022 Ops/s	36.8013 Ops/s	$\color{#35bf28}+0.82\%$
test_ppo_speed	32.6513ms	27.1185ms	36.8751 Ops/s	36.7407 Ops/s	$\color{#35bf28}+0.37\%$
test_reinforce_speed	27.8506ms	25.7362ms	38.8558 Ops/s	38.2748 Ops/s	$\color{#35bf28}+1.52\%$
test_iql_speed	0.1073s	68.8107ms	14.5326 Ops/s	15.5026 Ops/s	$\textbf{\color{#d91a1a}-6.26\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.1418ms	1.3963ms	716.2037 Ops/s	727.8516 Ops/s	$\color{#d91a1a}-1.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	9.5611ms	0.5221ms	1.9154 KOps/s	1.9008 KOps/s	$\color{#35bf28}+0.77\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	9.8178ms	0.4948ms	2.0209 KOps/s	2.0570 KOps/s	$\color{#d91a1a}-1.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.1054ms	1.4020ms	713.2453 Ops/s	699.9461 Ops/s	$\color{#35bf28}+1.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	8.7682ms	0.5191ms	1.9266 KOps/s	1.9592 KOps/s	$\color{#d91a1a}-1.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	9.1900ms	0.4968ms	2.0130 KOps/s	2.0137 KOps/s	$\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.2960ms	1.5896ms	629.1037 Ops/s	599.6369 Ops/s	$\color{#35bf28}+4.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8799ms	0.6496ms	1.5394 KOps/s	1.5204 KOps/s	$\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	8.5162ms	0.6372ms	1.5694 KOps/s	1.5867 KOps/s	$\color{#d91a1a}-1.09\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	1.7643ms	1.4574ms	686.1324 Ops/s	668.6514 Ops/s	$\color{#35bf28}+2.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	8.9815ms	0.5329ms	1.8765 KOps/s	1.9299 KOps/s	$\color{#d91a1a}-2.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6088ms	0.4901ms	2.0406 KOps/s	2.0343 KOps/s	$\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	1.7384ms	1.3689ms	730.5106 Ops/s	677.2337 Ops/s	$\textbf{\color{#35bf28}+7.87\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.5725ms	0.5178ms	1.9313 KOps/s	1.9035 KOps/s	$\color{#35bf28}+1.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5395ms	0.4885ms	2.0470 KOps/s	2.0599 KOps/s	$\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.2864ms	1.5622ms	640.1332 Ops/s	584.3619 Ops/s	$\textbf{\color{#35bf28}+9.54\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7924ms	0.6525ms	1.5326 KOps/s	1.4957 KOps/s	$\color{#35bf28}+2.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	9.0578ms	0.6386ms	1.5660 KOps/s	1.6003 KOps/s	$\color{#d91a1a}-2.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1164s	12.0215ms	83.1842 Ops/s	78.5964 Ops/s	$\textbf{\color{#35bf28}+5.84\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	22.8638ms	13.8512ms	72.1961 Ops/s	73.1702 Ops/s	$\color{#d91a1a}-1.33\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.3649ms	3.2604ms	306.7140 Ops/s	302.5646 Ops/s	$\color{#35bf28}+1.37\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1003s	9.9691ms	100.3104 Ops/s	80.4613 Ops/s	$\textbf{\color{#35bf28}+24.67\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.7114ms	13.4657ms	74.2626 Ops/s	73.4180 Ops/s	$\color{#35bf28}+1.15\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	11.2826ms	3.4428ms	290.4654 Ops/s	285.8706 Ops/s	$\color{#35bf28}+1.61\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1096s	12.5201ms	79.8717 Ops/s	92.3303 Ops/s	$\textbf{\color{#d91a1a}-13.49\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	16.1925ms	13.8197ms	72.3605 Ops/s	71.2571 Ops/s	$\color{#35bf28}+1.55\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	6.4365ms	3.5387ms	282.5875 Ops/s	281.5792 Ops/s	$\color{#35bf28}+0.36\%$

[agarwl]

I think that's the repository of @MaxASchwarzer and you need to ask him about this.

I can put a link to the pytorch dataset in the original offline RL repo.