-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid setting transforms in the buffer #150
Labels
enhancement
New feature or request
Comments
Any concerns on this? Otherwise I would add this to the next milestone and take care of it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, the
DataBuffer
receivestransform
andtarget_transform
and applies them when accessing an item. When deserializing theLearner
state, this requires resetting the transforms (seeDataBuffer.set_transforms
) and generally bloats the code of the buffer.I would propose to remove the transforms form the buffer and instead generalize the
_TransformedDataset
such that it can be applied toDataBuffer
s as well. This way, transforms are handled in a unified way by wrapping all relevant datasets in_TransformedDataset
in theLearner
.To make
_TransformedDataset
more general, instead of passingtransform
andtarget_transform
, we could pass a single transform that operates on the entire data point. When wrapping a buffer in_TransformedDataset
, we just need to make sure that the metadata are "ignored" (e.g.,transform=lambda (data_point, metadata): (my_transform(data_point), metadata)
).The text was updated successfully, but these errors were encountered: