Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ext/node): add missing _preloadModules hook #18447

Merged
merged 1 commit into from
Mar 27, 2023

Conversation

marvinhagemeister
Copy link
Contributor

@marvinhagemeister marvinhagemeister commented Mar 26, 2023

This internal node hook is used by libraries such as ts-node when used as a require hook node -r ts-node/register. That combination is often used with test frameworks like mocha or jasmine.

We had a reference to Module._preloadModules in our code, but the implementation was missing. While fixing this I also noticed that the fakeParent module that we create internally always threw because of the pathDirname check on the module id in the constructor of Mdoule. So this code path was probably broken for a while.

✖ ERROR: Error: Empty filepath.
    at pathDirname (ext:deno_node/01_require.js:245:11)
    at new Module (ext:deno_node/01_require.js:446:15)
    at Function.Module._resolveFilename (ext:deno_node/01_require.js:754:28)
    at Function.resolve (ext:deno_node/01_require.js:1015:19)

With the changes in this PR I'm now able to run mocha tests with ts-node in deno:

Screenshot 2023-03-26 at 22 47 46

@marvinhagemeister marvinhagemeister marked this pull request as ready for review March 26, 2023 20:59
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice fix!

@bartlomieju
Copy link
Member

I'm not so knowledgable on how ts-node works and I'm surprised you were able to get it running with this small change. I tried that in #17191, but didn't get very far - I guess it's used in different mode than I tried to use (the REPL).

@marvinhagemeister
Copy link
Contributor Author

@bartlomieju I might have just been lucky in picking the easier entry point. The ts-node/register entry point likely sidesteps a bunch of stuff that ts-node normally does.

@bartlomieju
Copy link
Member

@bartlomieju I might have just been lucky in picking the easier entry point. The ts-node/register entry point likely sidesteps a bunch of stuff that ts-node normally does.

Ah, that sounds reasonable!

@bartlomieju
Copy link
Member

@marvinhagemeister can you please run tools/format.js script to make CI checks happy?

@marvinhagemeister
Copy link
Contributor Author

marvinhagemeister commented Mar 26, 2023

@bartlomieju whoops my bad! Addressed the linting issues.

EDIT: Looks like CI failed again. I'm on it.
EDIT2: Fixed 🎉

@marvinhagemeister marvinhagemeister force-pushed the node-require-hooks branch 2 times, most recently from 11180d3 to 45b315f Compare March 26, 2023 23:49
throw new Error("Empty filepath.");
} else if (filepath === "") {
return ".";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node's path.dirname function doesn't throw when it receives an empty string like we did. Instead it returns ".". See https://github.com/nodejs/node/blob/38e6ac7b446bcf2be257fde6763d1da7d2104295/lib/path.js#L660-L661

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It appears this is correctly implemented in actual path.dirname() polyfill

ArrayPrototypePush(
paths,
ops.op_require_path_dirname(parent.filename),
parent?.filename ? ops.op_require_path_dirname(parent.filename) : ".",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -954,7 +974,7 @@ Module._extensions[".js"] = function (module, filename) {
const content = ops.op_require_read_file(filename);

if (StringPrototypeEndsWith(filename, ".js")) {
const pkg = ops.op_require_read_closest_package_json(filename);
const pkg = ops.op_require_read_package_scope(filename);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the real issue why it threw an error. It took me a while to find it. The difference is that the former throws an error and can lead to a panic. The latter returns an Option which our code kinda expects here. I think this was just a typo as the function in node refers to "package scope" as well: https://github.com/nodejs/node/blob/38e6ac7b446bcf2be257fde6763d1da7d2104295/lib/internal/modules/cjs/loader.js#L1307

@marvinhagemeister
Copy link
Contributor Author

marvinhagemeister commented Mar 26, 2023

@bartlomieju Addressing the CI error lead me down to a rabbit hole and it looks like you were much closer to success in your PR than you might've assumed! Turns out we had a couple of subtle deviations from node's behavior in our code. The new changes addresses these. Marked them with comments here. Let me know what you think.

EDIT: Rebased against latest changes in main

This internal node hook is used by libraries such as `ts-node`
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice investigation Marvin!

@bartlomieju bartlomieju merged commit 8c051db into denoland:main Mar 27, 2023
@marvinhagemeister marvinhagemeister deleted the node-require-hooks branch March 27, 2023 20:48
mmastrac pushed a commit that referenced this pull request Mar 31, 2023
This internal node hook is used by libraries such as `ts-node` when used
as a require hook `node -r ts-node/register`. That combination is often
used with test frameworks like `mocha` or `jasmine`.

We had a reference to `Module._preloadModules` in our code, but the
implementation was missing. While fixing this I also noticed that the
`fakeParent` module that we create internally always threw because of
the `pathDirname` check on the module id in the constructor of `Mdoule`.
So this code path was probably broken for a while.

```txt
✖ ERROR: Error: Empty filepath.
    at pathDirname (ext:deno_node/01_require.js:245:11)
    at new Module (ext:deno_node/01_require.js:446:15)
    at Function.Module._resolveFilename (ext:deno_node/01_require.js:754:28)
    at Function.resolve (ext:deno_node/01_require.js:1015:19)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants