Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only look up package.json type if module is node16/nodenext or file is in node_modules #58825

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

andrewbranch
Copy link
Member

@andrewbranch andrewbranch commented Jun 10, 2024

This walks back the heuristics added to non-Node.js module modes in #57896 a bit, based mostly on feedback in #58663. In 5.5-beta, an explicit package.json "type" or module-format-specific file extension started overriding "module": "commonjs" or "module": "esnext" in tsconfig.json. This impacted both type checking (relevant primarily for imports of node_modules dependencies, but with some effects on local files, e.g. with verbatimModuleSyntax) and emit (relevant for local project source files, if emit was enabled). The feedback in #58663 was that the effects on local files were unwelcome because the emitted files (whether from tsc or another tool) were going to be placed in some other context tsc didn’t know about, so the package.json tsc was finding during compilation shouldn’t be considered during the build.

With this PR, we only look up package.json "type" if --module is node16/nodenext (as we’ve always done) or if the file is inside node_modules.

Some testing and experimentation is still needed to ensure this doesn’t create inconsistencies with symlinked monorepos. This doesn’t seem to cause inconsistencies with workspaces-style monorepos, because module resolution takes the realpath of files in node_modules before we see if we need to look up a package.json "type". So in a project references example like

.
└── packages/
    ├── app/
    │   ├── node_modules/shared ↪︎
    │   ├── tsconfig.json
    │   ├── package.json
    │   └── index.ts
    └── shared/
        ├── dist/index.d.ts
        ├── src/index.ts
        ├── tsconfig.json
        └── package.json

When we build shared/tsconfig.json, we decide that shared/src/index.ts is not inside node_modules. Then, when we build app/tsconfig.json, even though we resolve app/node_modules/shared/dist/index.d.ts, we realpath it to shared/dist/index.d.ts and count it as not inside node_modules. Having this consistency between input and output files regardless of how they’re resolved is what matters most. The caveat is that the realpath and project references logic can cause the type checking of app/tsconfig.json during development to behave differently than it would after publishing/deploying into a situation where app/node_modules/shared is no longer a symlink, but a real node_modules package.

Fixes #54752
Fixes #50647

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 10, 2024
@andrewbranch
Copy link
Member Author

@typescript-bot test top400

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 11, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

@andrewbranch Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58825/merge:

Everything looks good!

@andrewbranch
Copy link
Member Author

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 11, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 11, 2024

Hey @andrewbranch, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162168/artifacts?artifactName=tgz&fileId=C1ACA7462D57C70E93C4D3280AA7732A4C19585806197BFA46B31559D77CD4B402&fileName=/typescript-5.6.0-insiders.20240611.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@andrewbranch
Copy link
Member Author

Perf should be unchanged or better, but

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 11, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@andrewbranch
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,725k (± 0.89%) 193,513k (± 0.90%) ~ 192,250k 195,779k p=0.230 n=6
Parse Time 1.30s (± 0.79%) 1.30s (± 0.64%) ~ 1.29s 1.31s p=0.923 n=6
Bind Time 0.72s 0.72s ~ ~ ~ p=1.000 n=6
Check Time 9.58s (± 0.35%) 9.58s (± 0.61%) ~ 9.53s 9.68s p=0.936 n=6
Emit Time 2.77s (± 0.37%) 2.76s (± 0.70%) ~ 2.74s 2.79s p=0.325 n=6
Total Time 14.37s (± 0.28%) 14.37s (± 0.44%) ~ 14.31s 14.45s p=0.871 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,109 944,109 ~ ~ ~ p=1.000 n=6
Types 407,049 407,049 ~ ~ ~ p=1.000 n=6
Memory used 1,222,155k (± 0.00%) 1,219,694k (± 0.00%) -2,462k (- 0.20%) 1,219,639k 1,219,782k p=0.005 n=6
Parse Time 8.09s (± 0.47%) 8.04s (± 0.18%) ~ 8.02s 8.06s p=0.064 n=6
Bind Time 2.24s (± 0.69%) 2.26s (± 0.40%) +0.02s (+ 0.89%) 2.25s 2.27s p=0.039 n=6
Check Time 36.63s (± 0.37%) 36.44s (± 0.43%) ~ 36.25s 36.67s p=0.054 n=6
Emit Time 18.07s (± 0.40%) 17.90s (± 0.55%) -0.17s (- 0.96%) 17.78s 18.03s p=0.013 n=6
Total Time 65.03s (± 0.26%) 64.64s (± 0.20%) -0.39s (- 0.60%) 64.46s 64.76s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 2,131,876 2,131,876 ~ ~ ~ p=1.000 n=6
Types 925,869 925,869 ~ ~ ~ p=1.000 n=6
Memory used 2,120,037k (± 0.00%) 2,116,210k (± 0.01%) -3,827k (- 0.18%) 2,116,098k 2,116,356k p=0.005 n=6
Parse Time 6.83s (± 0.17%) 6.69s (± 0.30%) -0.14s (- 2.05%) 6.66s 6.72s p=0.005 n=6
Bind Time 2.29s (± 0.18%) 2.32s (± 0.79%) +0.03s (+ 1.31%) 2.29s 2.34s p=0.016 n=6
Check Time 71.30s (± 0.38%) 71.26s (± 0.44%) ~ 70.70s 71.64s p=0.936 n=6
Emit Time 0.14s 0.14s ~ ~ ~ p=1.000 n=6
Total Time 80.56s (± 0.34%) 80.41s (± 0.37%) ~ 79.87s 80.74s p=0.378 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,424 1,230,425 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,178 261,178 ~ ~ ~ p=1.000 n=6
Memory used 2,376,991k (± 2.59%) 2,351,721k (± 0.06%) ~ 2,350,180k 2,353,055k p=0.689 n=6
Parse Time 4.98s (± 0.93%) 4.97s (± 0.89%) ~ 4.91s 5.04s p=0.810 n=6
Bind Time 1.90s (± 0.61%) 1.89s (± 0.33%) ~ 1.88s 1.90s p=0.070 n=6
Check Time 34.11s (± 0.39%) 34.12s (± 0.41%) ~ 33.98s 34.35s p=0.873 n=6
Emit Time 2.64s (± 0.79%) 2.66s (± 2.65%) ~ 2.57s 2.74s p=0.378 n=6
Total Time 43.65s (± 0.34%) 43.66s (± 0.26%) ~ 43.48s 43.82s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,424 1,230,425 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,178 261,178 ~ ~ ~ p=1.000 n=6
Memory used 2,427,658k (± 0.02%) 2,427,123k (± 0.05%) ~ 2,425,891k 2,428,927k p=0.298 n=6
Parse Time 5.17s (± 0.39%) 5.19s (± 1.04%) ~ 5.14s 5.26s p=1.000 n=6
Bind Time 1.68s (± 0.31%) 1.69s (± 0.75%) ~ 1.68s 1.71s p=0.418 n=6
Check Time 34.51s (± 0.29%) 34.57s (± 0.22%) ~ 34.49s 34.70s p=0.423 n=6
Emit Time 2.65s (± 2.76%) 2.70s (± 2.62%) ~ 2.58s 2.77s p=0.173 n=6
Total Time 44.02s (± 0.35%) 44.17s (± 0.19%) ~ 44.08s 44.31s p=0.065 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 259,181 259,182 +1 (+ 0.00%) ~ ~ p=0.001 n=6
Types 105,173 105,173 ~ ~ ~ p=1.000 n=6
Memory used 428,866k (± 0.03%) 428,867k (± 0.01%) ~ 428,827k 428,913k p=0.378 n=6
Parse Time 4.07s (± 0.65%) 4.09s (± 1.00%) ~ 4.03s 4.15s p=0.517 n=6
Bind Time 1.63s (± 0.74%) 1.62s (± 1.49%) ~ 1.59s 1.65s p=0.622 n=6
Check Time 22.27s (± 0.67%) 22.38s (± 0.30%) ~ 22.27s 22.46s p=0.228 n=6
Emit Time 1.74s (± 1.01%) 1.72s (± 1.41%) ~ 1.69s 1.76s p=0.126 n=6
Total Time 29.72s (± 0.47%) 29.80s (± 0.39%) ~ 29.60s 29.89s p=0.376 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,796k (± 0.04%) 369,638k (± 0.03%) -158k (- 0.04%) 369,515k 369,775k p=0.045 n=6
Parse Time 3.46s (± 1.35%) 3.46s (± 0.41%) ~ 3.44s 3.48s p=0.936 n=6
Bind Time 1.95s (± 0.96%) 1.95s (± 2.12%) ~ 1.89s 2.01s p=0.807 n=6
Check Time 19.36s (± 0.28%) 19.40s (± 0.19%) ~ 19.35s 19.44s p=0.148 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 24.77s (± 0.31%) 24.82s (± 0.16%) ~ 24.74s 24.85s p=0.224 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,852,976 2,852,976 ~ ~ ~ p=1.000 n=6
Types 967,867 967,867 ~ ~ ~ p=1.000 n=6
Memory used 3,025,016k (± 0.00%) 3,020,070k (± 0.00%) -4,946k (- 0.16%) 3,019,976k 3,020,204k p=0.005 n=6
Parse Time 13.82s (± 0.16%) 13.65s (± 0.40%) -0.18s (- 1.30%) 13.57s 13.73s p=0.005 n=6
Bind Time 4.27s (± 2.62%) 4.17s (± 0.25%) ~ 4.16s 4.19s p=0.406 n=6
Check Time 74.97s (± 2.49%) 73.60s (± 0.32%) ~ 73.27s 73.96s p=0.066 n=6
Emit Time 22.53s (± 8.36%) 23.90s (± 0.77%) ~ 23.69s 24.12s p=0.054 n=6
Total Time 115.59s (± 0.17%) 115.32s (± 0.31%) ~ 114.88s 115.77s p=0.173 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,914 266,914 ~ ~ ~ p=1.000 n=6
Types 108,685 108,685 ~ ~ ~ p=1.000 n=6
Memory used 411,894k (± 0.02%) 411,499k (± 0.02%) -396k (- 0.10%) 411,379k 411,655k p=0.005 n=6
Parse Time 3.82s (± 1.09%) 3.83s (± 0.59%) ~ 3.80s 3.86s p=0.627 n=6
Bind Time 1.66s (± 0.99%) 1.69s (± 0.65%) +0.03s (+ 2.11%) 1.67s 1.70s p=0.007 n=6
Check Time 17.01s (± 0.45%) 16.98s (± 0.42%) ~ 16.84s 17.03s p=0.688 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.49s (± 0.35%) 22.51s (± 0.38%) ~ 22.34s 22.58s p=0.809 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,429 523,429 ~ ~ ~ p=1.000 n=6
Types 177,955 177,955 ~ ~ ~ p=1.000 n=6
Memory used 461,890k (± 0.02%) 461,564k (± 0.07%) ~ 461,216k 461,953k p=0.093 n=6
Parse Time 3.15s (± 0.68%) 3.15s (± 0.55%) ~ 3.12s 3.17s p=0.872 n=6
Bind Time 1.18s (± 0.46%) 1.19s (± 0.69%) ~ 1.18s 1.20s p=0.859 n=6
Check Time 18.21s (± 0.20%) 18.14s (± 0.83%) ~ 17.95s 18.32s p=0.568 n=6
Emit Time 0.00s (±244.70%) 0.00s ~ ~ ~ p=0.405 n=6
Total Time 22.55s (± 0.24%) 22.47s (± 0.66%) ~ 22.30s 22.65s p=0.520 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@andrewbranch andrewbranch force-pushed the package-json-type-only-in-node_modules branch from 2677b01 to 6208d2f Compare June 17, 2024 18:55
@typescript-bot
Copy link
Collaborator

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

@andrewbranch andrewbranch marked this pull request as ready for review June 17, 2024 22:44
@typescript-bot typescript-bot added For Milestone Bug PRs that fix a bug with a specific milestone and removed For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 17, 2024
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need these new ModuleSpecifierResolutionHost members to be public because we automatically attach them or have fallbacks, right? Because these'll be missing from a user-provided host object. We probably wannt mark 'em as optional and handle appropriately if they're non-public.

@andrewbranch
Copy link
Member Author

We don't need these new ModuleSpecifierResolutionHost members to be public

@weswigham ModuleSpecifierResolutionHost itself is internal.

src/compiler/checker.ts Outdated Show resolved Hide resolved
@andrewbranch
Copy link
Member Author

There’s a knip failure but it’s public API?

@weswigham
Copy link
Member

ModuleSpecifierResolutionHost itself is internal.

The specifics are internal, yes, but we expose and consume public objects and interfaces that we use as and expect to be ModuleSpecifierResolutionHosts which cannot be correctly implemented without these being made public in some way. There's a definite reason almost everything on ModuleSpecifierResolutionHost is optional. For example, Program has a second (@internal) declaration declaring it extends ModuleSpecifierResolutionHost. If there are any 3rd-party Programs (like our LS or builder construct internally to orchestrate builds, but external), they won't know they need these new methods on their Program to satisfy ModuleSpecifierResolutionHost uses, and since they're required, we'll just crash at runtime when they're inevitably missing.

@andrewbranch
Copy link
Member Author

andrewbranch commented Jul 1, 2024

There are 43 non-optional internal methods and properties declared on the public Program interface declaration—we haven’t made it possible to implement a third-party Program up from its public interface.

@andrewbranch
Copy link
Member Author

@weswigham any other feedback on the API (or any of the PR)? I’d like to get this in soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
Status: Needs merge
5 participants