Allow consumers of Compiler API to override getDirectoryToWatchFailedLookupLocation

[dmichon-msft]

🔍 Search Terms

getDirectoryToWatchFailedLookupLocation, watch, resolver, compiler API

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This isn't a request to add a new utility type: https://github.com/microsoft/TypeScript/wiki/No-New-Utility-Types
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Consumers of TypeScript's compiler API are typically in a better position to understand the nuances of the layout of the repository and take advantage of that knowledge to optimize file watcher performance than any generic implementation in TypeScript.

For large monorepos especially, TypeScript's implementation of getDirectoryToWatchFailedLookupLocation does not scale well and results in watching tons of excess files and directories.

If TypeScript were to access this method (and possible also getDirectoryToWatchFromFailedLookupLocationDirectory) via the CompilerHost interface (defaulting to the current implementation), consumers of the compiler API who work with customized file system layouts could better instruct the compiler on how to efficiently watch their filesystem. This would also help with exotic file system scenarios (e.g. network file systems, virtual filesystems).

The current method:
https://github.com/microsoft/TypeScript/blob/b258429aaaa6206fd5f911360b094ca626544ce9/src/compiler/resolutionCache.ts#L347C17-L394

📃 Motivating Example

I work in a large monorepo that has a small number of root folder (>1) and hundreds of projects under each root. Frequently projects under one root reference projects under another root, and under the default implementation this results in TypeScript watching hundreds of projects instead of the handful that are relevant to its execution.

On Linux especially, the decision to ask for a recursive watcher by default results in significant over-allocation of file and directory watchers, since the platform has no native support and therefore it results in allocating a non-recursive watcher for each and every directory in the tree.

💻 Use Cases

1. What do you want to use this for?

Integrating with Heft for use in Rush monorepos, particularly on GitHub codespaces (which are Linux-based)

2. What shortcomings exist with current approaches?

The current workaround is brittle and will need to be updated any time I update to a newer version of TypeScript.

3. What workarounds are you using in the meantime?

I have locally patched the installation of TypeScript in my monorepo to override this function with an alternative implementation that non-recursively watches the immediate parent folder of the failed resolution location.

diff --git a/lib/typescript.js b/lib/typescript.js
index fe732a6cb262712d4d79b917b7fa0f0eee9d58b4..e2534aa5dc62b7ea70b337687ed7d194b0ccf1e9 100644
--- a/lib/typescript.js
+++ b/lib/typescript.js
@@ -124353,6 +124353,13 @@ ${lanes.join("\n")}
     if (failedLookupPathComponents.length <= perceivedOsRootLength + 1)
       return void 0;
     const nodeModulesIndex = failedLookupPathComponents.indexOf("node_modules");
+
+    if (process.env.CODESPACE_NAME) {
+      // Hack to prevent excessive directory watching on Linux.
+      // In a Rush repo, the node_modules folder is considered immutable while running, so don't watch it.
+      // Linux does not support recursive watching, so always only watch the immediate parent folder.
+      return nodeModulesIndex >= 0 || failedLookupComponents.length < 3 ? void 0 : getDirectoryOfFailedLookupWatch(failedLookupComponents, failedLookupPathComponents, failedLookupComponents.length - 1, true);
+    }
     if (nodeModulesIndex !== -1 && nodeModulesIndex + 1 <= perceivedOsRootLength + 1)
       return void 0;
     if (isInDirectoryPath(rootPathComponents, failedLookupPathComponents)) {
@@ -124376,6 +124383,13 @@ ${lanes.join("\n")}
     );
   }
   function getDirectoryToWatchFromFailedLookupLocationDirectory(dirComponents, dirPathComponents, dirPathComponentsLength, perceivedOsRootLength, nodeModulesIndex, rootPathComponents) {
+    if (process.env.CODESPACE_NAME) {
+      // Hack to prevent excessive directory watching on Linux.
+      // In a Rush repo, the node_modules folder is considered immutable while running, so don't watch it.
+      // Linux does not support recursive watching, so always only watch the immediate parent folder.
+      return nodeModulesIndex >= 0 || failedLookupComponents.length < 3 ? void 0 : getDirectoryOfFailedLookupWatch(failedLookupComponents, failedLookupPathComponents, failedLookupComponents.length - 1, true);
+    }
+
     if (nodeModulesIndex !== -1) {
       return getDirectoryOfFailedLookupWatch(dirComponents, dirPathComponents, nodeModulesIndex + 1);
     }

[RyanCavanaugh]

@sheetalkamat thoughts?

[dmichon-msft]

Related #58866.

[sheetalkamat]

I think better approach is to handle this like in #58866 than adding API for this as this is not just resolutionCache issues but also needs to handle closed script info's node_modules folder etc as well