Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add waker to StreamResource #4293

Merged
merged 13 commits into from
Mar 11, 2020

Conversation

jsouto18
Copy link
Contributor

@jsouto18 jsouto18 commented Mar 8, 2020

This Pull Request serves to present a WIP solution for the issues #4284 and #3390.
I've traced the cause of #4284 to the poll_read task that was getting stuck on Poll::Pending. It was never being woken up due to the already closed underlying connection.
This WIP solution implements the same waking solution used on TcpListenerResource under //cli/ops/net.rs for the StreamResourceHolder struct.

Currently only the net.rs tcp resources function properly.

@jsouto18
Copy link
Contributor Author

jsouto18 commented Mar 8, 2020

@ry @bartlomieju (review)

cli/ops/io.rs Outdated
Comment on lines 219 to 231
let nread = match resource_holder
.resource
.poll_read(cx, &mut buf.as_mut()[..])
{
Poll::Ready(t) => {
resource_holder.untrack_task();
t
}
Poll::Pending => {
resource_holder.track_task(cx)?;
return Poll::Pending;
}
}?;
Copy link
Member

@bartlomieju bartlomieju Mar 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ry, @piscisaureus and I had a discussion about this solution; it looks good on the surface; but it effectively prevents having multiple ops reading from single resource. Although in most cases we only want to have a single op, there are some cases where having multiple ops reading from single resource is desirable.

Ideally we would track multiple tasks for single resource, but Waker is very simple struct that has no PartialEq or any other trait implementation that would allow to discriminate wakers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll discuss it more tonight

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bartlomieju I pushed some small changes to the StreamResourceWrapper to make it capable of waking multiple tasks. It just a suggestion to help the discussion.

@ry
Copy link
Member

ry commented Mar 9, 2020

I see that @keroxp has a test in #4284 - do we have a smaller example we can integrate into our test suite? This PR does not add any tests.

@jsouto18
Copy link
Contributor Author

jsouto18 commented Mar 9, 2020

@ry I’ll add some tests ofc. I just wanted to know what you guys thought about this solution first.

@bartlomieju
Copy link
Member

Another test case (in std/) can be found in this PR: #3679

@ry
Copy link
Member

ry commented Mar 9, 2020

We should get a test that involves only TCP code (not the http server) and just does listen, accept, read.

@jsouto18 Thanks for the patch - but we want to look into this in more detail.

@bartlomieju
Copy link
Member

@jsouto18 can you incorporate one of tests mentioned above?

@jsouto18
Copy link
Contributor Author

@bartlomieju I've implemented #4284 example as a test. I use a new timeout arg to test if the process is hanging.

@bartlomieju
Copy link
Member

bartlomieju commented Mar 11, 2020

@bartlomieju I've implemented #4284 example as a test. I use a new timeout arg to test if the process is hanging.

That's a bit too complex; can you revert those changes and used this boiled down reproduction (put it in cli/js/tests/net_test.ts):

unitTest(
  {
    perms: { net: true }
  },
  async function netHangsOnClose() {
    let acceptedConn: Deno.Conn;
    const resolvable = createResolvable();

    async function iteratorReq(listener: Deno.Listener): Promise<void> {
      const p = new Uint8Array(10);
      const conn = await listener.accept();
      acceptedConn = conn;
      
      while (true) {
        const nread = await conn.read(p);
        console.log("read", nread, "bytes");
        if (nread === Deno.EOF) {
          break;
        }
        const nwritten = await conn.write(new Uint8Array([1, 2, 3]));
        console.log("written", nwritten, "bytes");
      }

      resolvable.resolve();
    }

    const addr = { hostname: "127.0.0.1", port: 4500 };
    const listener = Deno.listen(addr);
    iteratorReq(listener);
    const conn = await Deno.connect(addr);
    await conn.write(new Uint8Array([1, 2, 3, 4]));
    const buf = new Uint8Array(10);
    const nread = await conn.read(buf);
    console.log("main read", nread);
    console.log("accepted", acceptedConn!);
    acceptedConn!.close();
    listener.close();
    await resolvable;
  }
);

It does hang on master for me

@jsouto18
Copy link
Contributor Author

@bartlomieju the changes are reverted and I've added the new test. 😃
The reason for the timeout implementation was to make sure an error was thrown instead of having the test suite hang indefinitely.

Copy link
Member

@ry ry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsouto18 Thank you for this patch and thank you @bartlomieju for the succinct test.

I think there might be a deeper design issue here - but the solution isn't clear to me now - maybe we should be tracking in a more general way which ops operate on which resources. This makes me hesitate on this patch, because it introduces infrastructure for only stream resources.

That said, it fixes bugs and we have a new test that we can refactor around, so I will land it now.

@ry ry merged commit fb5c314 into denoland:master Mar 11, 2020
dubiousjim added a commit to dubiousjim/deno that referenced this pull request Mar 13, 2020
* denoland/master:
  Remove doc strings from cli/js TS files (denoland#4329)
  upgrade: Rust 1.42.0 (denoland#4331)
  Enable std tests in debug mode (denoland#4332)
  fix: Node polyfill fsAppend rework (denoland#4322)
  v0.36.0
  Add waker to StreamResource to fix hang on close bugs (denoland#4293)
  reorg: Deno global initialization (denoland#4317)
  move compiler API tests to integration tests (denoland#4319)
  support permission mode in mkdir (denoland#4286)
  Stricter permissions for Deno.makeTemp* (denoland#4318)
  reorg: remove dispatch.ts, move signals, factor out web utils (denoland#4316)
  reorg: cli/js/compiler/, move more API to cli/js/web/ (denoland#4310)
  Improve dprint config (denoland#4314)
  doc(cli/flags): Reduce empty lines in help messages (denoland#4312)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants