Skip to content

v0.5.0: Dynamic specs, envs with non-tensor data and replay buffer checkpointers

Latest
Latest
Compare
Choose a tag to compare
@vmoens vmoens released this 30 Jul 22:36
v0.5.0
9565398

What's Changed

This new release makes it possible to run environments that output non-tensor data. #1944

We also introduce dynamic specs, allowing environments to change the size of the observations / actions during the
course of a rollout. This feature is compatible with parallel environment and collectors! #2143

Additionally, it is now possible to update a Replay Buffer in-place by assigning values at a given index. #2224

Finally, TorchRL is now compatible with Python 3.12 (#2282, #2281).

As always, a huge thanks to the vibrant OSS community that helps us developt this library!

New algorithms

Features

Bug fixes

  • [BugFix,Feature] Allow non-tensor data in envs by @vmoens in #1944
  • [BugFix] Allow zero alpha value for PrioritizedSampler by @albertbou92 in #2164
  • [BugFix] Expose MARL modules by @vmoens in #2321
  • [BugFix] Fit vecnorm out_keys by @vmoens in #2157
  • [BugFix] Fix Brax by @vmoens in #2233
  • [BugFix] Fix OOB sampling in PrioritizedSliceSampler by @vmoens in #2239
  • [BugFix] Fix VecNorm test in test_collectors.py by @vmoens in #2162
  • [BugFix] Fix to in MultiDiscreteTensorSpec by @Quinticx in #2204
  • [BugFix] Fix and test PRB priority update across dims and rb types by @vmoens in #2244
  • [BugFix] Fix another ctx test by @vmoens in #2284
  • [BugFix] Fix async gym env with non-sync resets by @vmoens in #2170
  • [BugFix] Fix async gym when all reset by @vmoens in #2144
  • [BugFix] Fix brax wrapping by @vmoens in #2190
  • [BugFix] Fix collector tests where device ordinal is needed by @vmoens in #2240
  • [BugFix] Fix collectors with non tensors by @vmoens in #2232
  • [BugFix] Fix done/terminated computation in slice samplers by @vmoens in #2213
  • [BugFix] Fix info reading with async gym by @vmoens in #2150
  • [BugFix] Fix isaac - bis by @vmoens in #2119
  • [BugFix] Fix lib tests by @vmoens in #2218
  • [BugFix] Fix max value within buffer during update priority by @vmoens in #2242
  • [BugFix] Fix max-priority update by @vmoens in #2215
  • [BugFix] Fix non-tensor passage in _StepMDP by @vmoens in #2260
  • [BugFix] Fix non-tensor passage in _StepMDP by @vmoens in #2262
  • [BugFix] Fix prefetch in samples without replacement - .sample() compatibility issues by @vmoens in #2226
  • [BugFix] Fix sampling in NonTensorSpec by @vmoens in #2172
  • [BugFix] Fix sampling of values from NonTensorSpec by @vmoens in #2169
  • [BugFix] Fix slice sampler end computation at the cursor place by @vmoens in #2225
  • [BugFix] Fix sliced PRB when only traj is provided by @vmoens in #2228
  • [BugFix] Fix strict length in PRB+SliceSampler by @vmoens in #2202
  • [BugFix] Fix strict_length in prioritized slice sampler by @vmoens in #2194
  • [BugFix] Fix tanh normal mode by @vmoens in #2198
  • [BugFix] Fix tensordict private imports by @vmoens in #2275
  • [BugFix] Fix test_specs.py by @vmoens in #2214
  • [BugFix] Fix torch 2.3 compatibility of padding indices by @vmoens in #2216
  • [BugFix] Fix truncated normal by @vmoens in #2147
  • [BugFix] Fix typo in weight assignment in PRB by @vmoens in #2241
  • [BugFix] Fix update_priority generic signature for Samplers by @vmoens in #2252
  • [BugFix] Fix vecnorm state-dicts by @vmoens in #2158
  • [BugFix] Global import of optional library by @matteobettini in #2217
  • [BugFix] Gym async with _reset full of True by @vmoens in #2145
  • [BugFix] MLFlow logger by @GJBoth in #2152
  • [BugFix] Make DMControlEnv aware of truncated signals by @vmoens in #2196
  • [BugFix] Make _reset follow done shape by @matteobettini in #2189
  • [BugFix] EnvBase._complete_done to complete "terminated" key properly by @kurtamohler in #2294
  • [BugFix] LazyTensorStorage only allocates data on the given device by @matteobettini in #2188
  • [BugFix] done = done | truncated in collector by @vmoens in #2333
  • [BugFix] buffer iter for samplers without replacement + prefetch by @JulianKu in #2185
  • [BugFix] buffer __iter__ for samplers without replacement + prefetch by @JulianKu in #2178
  • [BugFix] missing deprecated kwargs by @fedebotu in #2125

Docs

Performance

  • [Performance, Refactor, BugFix] Faster loading of uninitialized storages by @vmoens in #2221
  • [Performance] consolidate TDs in ParallelEnv without buffers by @vmoens in #2231

Others

New Contributors

Full Changelog: v0.4.0...v0.5.0

Contributors

kurtamohler, N00bcak, and 14 other contributors
Assets 17
Loading