Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): streams #12596

Merged
merged 16 commits into from
Nov 9, 2021
Merged

feat(core): streams #12596

merged 16 commits into from
Nov 9, 2021

Conversation

AaronO
Copy link
Contributor

@AaronO AaronO commented Oct 29, 2021

This allows resources to be "streams" by implementing read/write/shutdown, returning None if not implemented and Some(future) otherwise.

These streams are implicit since their nature (read/write/duplex) isn't known until called, but we could easily add another method to explicitly tag resources as streams.

op_read/op_write/op_shutdown are now builtin ops provided by deno_core

Note: this current implementation is simple & straightforward but it results in an additional alloc per read/write call

Closes #12556

@AaronO AaronO changed the title Core/streams feat(core): streams Oct 29, 2021
/// All objects that can be store in the resource table should implement the
/// `Resource` trait.
/// TODO(@AaronO): investigate avoiding alloc on read/write/shutdown
Copy link
Member

@piscisaureus piscisaureus Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pretty certain that boxing the returned future is unavoidable with trait methods.
It'd be nice though if we could avoid double boxing.

core/resources.rs Outdated Show resolved Hide resolved
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about sync APIs?

runtime/js/12_io.js Show resolved Hide resolved
@AaronO
Copy link
Contributor Author

AaronO commented Oct 30, 2021

What about sync APIs?

Since sync APIs are only used for file IO (on StdFileResource) I was planning to rename them to op_fread_sync, op_fwrite_sync, I don't think those sync APIs should be in core given that they're only used for files and we don't have any other obvious implementors for now.

This would leave them out from IO metrics in core, but this is easy to reconsider down the line

@bartlomieju
Copy link
Member

What about sync APIs?

Since sync APIs are only use for file IO (on StdFileResource) I was planning to rename them to op_fread_sync, op_fwrite_sync, I don't think those sync APIs should be in core

Okay, keep in mind that StdFileResource is used for stdio too.

Copy link
Member

@piscisaureus piscisaureus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't see why these ops should return Option<AsyncResult<_>> rather than AsyncResult<_>.
IMO that's just un-ergonomic and the "op not supported by this resource type" case really doesn't need any optimization.
Or am I missing something?

@AaronO
Copy link
Contributor Author

AaronO commented Nov 9, 2021

I still don't see why these ops should return Option<AsyncResult<_>> rather than AsyncResult<_>. IMO that's just un-ergonomic and the "op not supported by this resource type" case really doesn't need any optimization. Or am I missing something?

I think the "ergonomics" of the trait weren't a priority for a first pass given the limited amount of callers. The plan was to tweak and evolve this. I still generally don't like the idea that we have to peek into errors to know if the stream was readable or not. But regardless I've dropped the Option<...> from the trait in the latest commit

Copy link
Member

@piscisaureus piscisaureus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@AaronO AaronO merged commit 375ce63 into denoland:main Nov 9, 2021
@AaronO AaronO deleted the core/streams branch November 9, 2021 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

StreamResource in core
3 participants