Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[api-extractor] Reuse TypeScript's module-resolution results to avoid realpath/lstat storm when resolving external package names

Open
#5,892 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Quiet
Tech stack
node.js, typescript

Research direction

Start at DeclarationReferenceGenerator._getPackageName, then inspect Collector, AstSymbolTable, and TypeScriptInternals.ts for program-resolution access. Include module and type-reference resolutions, verify packageId and resolvedFileName matching, and run the acceptance-test projects to confirm byte-identical API reports while preserving the filesystem fallback.

Written by the indexing model from the issue text.

Description

Summary

During analysis, DeclarationReferenceGenerator._getPackageName(sourceFile) calls PackageJsonLookup.tryLoadNodePackageJsonFor(sourceFile.fileName) once per external source file. That path (PackageJsonLookup._tryLoadNodePackageJsonInner) calls FileSystem.getRealPath() (Node fs.realpathSync → an lstat per path segment, following symlinks) before the content cache can help, and the upward folder walk in _tryGetPackageFolderFor does it at each directory level. In pnpm workspaces (a node_modules symlink farm) this produces a large lstat storm and shows up as a significant chunk of api-extractor CPU time in profiles.

This was noticed while profiling the type-check optimization in https://github.com/microsoft/rushstack/pull/5891 — after removing the whole-program semantic check, realpathSynclstat inside FileSystem.getRealPathtryLoadNodePackageJsonFor was the next-largest chunk.

Insight

TypeScript already resolved every one of these modules during program construction and recorded the answer. Each ResolvedModuleFull carries:

  • resolvedFileName — the already-realpath'd target (TS paid the lstat cost once, cached), and
  • packageId.name — the bare package name, which is exactly what _getPackageName reconstructs via the filesystem walk (packageId.subModuleName holds the sub-path separately).
Proof of concept

Probing the @rushstack/mcp-server program via the (internal) program.forEachResolvedModule(cb):

resolutions: total=1111, withPackageId=1103, uniqueFiles=258
  @modelcontextprotocol/sdk  <=  @modelcontextprotocol/sdk/dist/esm/server/mcp.d.ts
  zod                        <=  zod/index.d.cts
  ...

99.3% of resolutions carry packageId, and packageId.name came back as the bare package name even for non-index files.

Proposed design
  1. Build a Map<resolvedFileName, packageId.name> once (e.g. off Collector/AstSymbolTable) via the internal program.forEachResolvedModule(...). Type-reference-directive resolutions (forEachResolvedTypeReferenceDirective) also carry packageId and should be included.
  2. In _getPackageName, consult the map first (keyed by sourceFile.fileName) — zero filesystem access for the common case.
  3. Fall back to tryLoadNodePackageJsonFor on a miss (the ~0.7% without packageId, or files reached via other means), preserving today's exact behavior.

forEachResolvedModule / getResolvedModule are TypeScript internals (not in the public .d.ts), but api-extractor already accesses TS internals via TypeScriptInternals.ts, so this is consistent with existing patterns.

Caveats to validate
  • packageId.name semantics — the TS doc comment hints it may include a subpath in some cases; the PoC showed bare names, but this needs verification against @types packages and deep imports vs. the existing packageJson.name behavior.
  • Key matchingresolvedFileName must equal the program's sourceFile.fileName (generally true; preserveSymlinks is the edge case).
  • Correctness — must produce byte-identical API reports across all acceptance-test projects. The fast-path-with-fallback design keeps this low-risk.
Related
  • Follow-up to #5891 (type-check pass optimization).
  • Also worth checking SourceMapper.getRealPath usage for the same treatment.
Dominant language
TypeScript
Stars
6.5k
Forks
708
Avg merge
5d 19h
Merged PRs (30d)
48

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/rushstack

All issues in microsoft/rushstack

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.