Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: disable 'test-http-content-length.js` test #20344

Merged
merged 2 commits into from
Sep 4, 2023

Conversation

bartlomieju
Copy link
Member

…d of Transfer-Encoding: chunked (#20127)"

This reverts commit c2547ba.

After merging that PR there's quite a lot of flakiness in Node compat tests that
fail like so:

Node.js compatibility ... Node.js compatibility "parallel/test-http-content-length.js" => ./cli/tests/node_compat/test.ts:58:11
error: AssertionError: Failed assertion: "parallel/test-http-content-length.js" failed:

error: Uncaught (in promise) TypeError: error sending request for url (https://localhost:50029/multiple-writes): connection error: Connection reset by peer (os error 54)
    at async Promise.all (index 0)
    at async node:http:432:26


You can repeat only this test with the command:

  ./target/debug/deno test -A cli/tests/node_compat/test.ts -- parallel/test-http-content-length.js

    throw new AssertionError(msg);
          ^
    at assert (file:https:///Users/runner/work/deno/deno/test_util/std/testing/asserts.ts:141:11)
    at fail (file:https:///Users/runner/work/deno/deno/test_util/std/testing/asserts.ts:602:3)
    at fn (file:https:///Users/runner/work/deno/deno/cli/tests/node_compat/test.ts:122:9)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async innerWrapped (ext:cli/40_testing.js:483:5)
    at async Object.outerWrapped [as fn] (ext:cli/40_testing.js:428:14)
    at async TestContext.step (ext:cli/40_testing.js:1241:22)
    at async runTest (file:https:///Users/runner/work/deno/deno/cli/tests/node_compat/test.ts:58:3)
    at async file:https:///Users/runner/work/deno/deno/test_util/std/async/pool.ts:76:11

 FAILURES 

Node.js compatibility ... Node.js compatibility "parallel/test-http-content-length.js" => ./cli/tests/node_compat/test.ts:58:11

CC @osddeitf

@osddeitf
Copy link
Contributor

osddeitf commented Sep 2, 2023

The problem may lie in the node-compat test I added.
The test was problematic when I first added it from node test suite, it couldn't fail even if there's AssertionError, so I tried to modify it.
If the test fails often and affect the automation test, please consider revert only the test.

@osddeitf
Copy link
Contributor

osddeitf commented Sep 2, 2023

The previous behavior of Deno node:http is to send request when the class ClientRequest is constructed.
What I modified is to delay the request until the first body chunk get sent, or when request.end().
That also fixes the methods setHeader(), flushHeader() and removeHeader().

@bartlomieju
Copy link
Member Author

The problem may lie in the node-compat test I added. The test was problematic when I first added it from node test suite, it couldn't fail even if there's AssertionError, so I tried to modify it. If the test fails often and affect the automation test, please consider revert only the test.

@osddeitf could you point me to the relevant part that should fail but didn't? Maybe there's another bug hiding there that I could fix.

@osddeitf
Copy link
Contributor

osddeitf commented Sep 4, 2023

This is my fix attempt:

  • 724c966: assert only relevant headers.
  • 1a98037: close http server when there's an AssertionError (if the server still running, the test can't stop).
  • aca7b73: the test couldn't fail so I store the AssertionError and throw when the server is closed.

I'll pull the latest code and try to rerun the test again, before, it worked totally normal for me (no flakiness), it may also because of other changes to Node compatibility layer.

@bartlomieju
Copy link
Member Author

@osddeitf ideally we'd just use test file from Node without any changes, if it fails because of discrepancies in sent headers we can remove unnecessary headers from being sent.

@osddeitf
Copy link
Contributor

osddeitf commented Sep 4, 2023

I know, that's why I changed that in the first commit mentioned above. The test passed as I expected.
But to make sure I tried to switch to earlier version of Deno to test if it would fail.
And instead of fail, the test do print the AssertionError, but the http server is not closed because countdown never reached 0, the test just stuck there instead of stop.
Hence the second commit.

@bartlomieju
Copy link
Member Author

Alright, let's then work on fixing these assertion errors!

@osddeitf
Copy link
Contributor

osddeitf commented Sep 4, 2023

Thank you! We still have some time till v1.37, I'll do my part too when I have some free time.

@bartlomieju
Copy link
Member Author

Thank you! We still have some time till v1.37, I'll do my part too when I have some free time.

Sounds good, I will then disable the test for now as it is causing flakes on CI

@bartlomieju bartlomieju changed the title Revert "fix(node/http): correctly send Content-length header instea… chore: disable 'test-http-content-length.js` test Sep 4, 2023
Copy link
Contributor

@mmastrac mmastrac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bartlomieju bartlomieju merged commit 2a1ba27 into denoland:main Sep 4, 2023
13 checks passed
@bartlomieju bartlomieju deleted the revert_20_217 branch September 4, 2023 21:23
@osddeitf
Copy link
Contributor

osddeitf commented Sep 6, 2023

@bartlomieju let me report on what I found (again, sorry if it's too long):

  • The error message indicates a Promise.all() rejection with index 0, I was able to pinpoint the error at ClientRequest.end() in node:http:

      (async () => {
        try {
          const [res, _] = await Promise.all([
            core.opAsync("op_fetch_send", this._req.requestRid),

    Note that the promise is awaited, and it is wrapped inside an async IIFE.

  • Deno implementation of http.Server:

    Note that the server is abruptly closed.

  • Node implementation of http.Server (kinda complicated):

    Note that the server gracefully shutdown, waiting for connections to close.

  • Error message Connection reset by peer (os error 54), indicates that the request did get to the server, but the server suddenly closed.

So I surmise that inside the test, the server get closed (when countdown reached 0) before res.end() complete, and as a result, the request get closed abrubtly.

@osddeitf
Copy link
Contributor

osddeitf commented Sep 6, 2023

In Nodejs docs server.close([callback]):

Stops the server from accepting new connections. See net.Server.close().

So I think this is indeed an issue of Server.close() in Deno implementation.

@bartlomieju
Copy link
Member Author

@osddeitf no worries. Nice investigation. Let's wait for #20387 to land and then it should be an easy fix 👌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants