Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support java streams on ndjson read and write #370

Closed
nmaguiar opened this issue Apr 11, 2022 · 0 comments
Closed

Support java streams on ndjson read and write #370

nmaguiar opened this issue Apr 11, 2022 · 0 comments

Comments

@nmaguiar
Copy link
Collaborator

When using io.writeLineNDJSON and io.readLinesNDJSON the ndjson files can be unusually large so it's useful to use gzip streams, to do something like:

var is = io.readFileGzipStream("data.ndjson.gz")
io.readFilesNDJSON(is, r => sprint(r))
is.close()
@nmaguiar nmaguiar self-assigned this Apr 11, 2022
@nmaguiar nmaguiar added this to Backlog in Continuous Enhancement via automation Apr 11, 2022
nmaguiar added a commit that referenced this issue Apr 11, 2022
@nmaguiar nmaguiar mentioned this issue Apr 12, 2022
Merged
Continuous Enhancement automation moved this from Backlog to Done Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

1 participant