Importing the npm typescript package costs 17–29 min / 7–10 GB to compile (whole 9 MB typescript.js codegen'd)

Aperta
  • Qualcosa non va?
#13,976 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
rust, typescript
Ambito
compilers, performance

Direzione di ricerca

Start by reproducing the typescript import probe in the issue and compare its compile time and memory use with the supplied measurements. Then investigate the listed options: reachability before codegen, a content keyed object cache, or cheaper codegen for large modules; the issue says these approaches are ideas that still need measuring. Done means reducing compile cost for programs importing typescript while preserving the Node-matching runtime behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

package-audit performance

Summary

A program that imports the npm typescript package (tsc used as a library) takes 17–29 minutes and 7–10 GB peak RSS to compile, and the binary is 78 MB. Another ~18 minutes in the program-facts stage comes on top by default (#13974). The program itself runs correctly and matches Node byte-for-byte; the problem is entirely compile cost.

This only affects programs that import typescript at run time: linters, code generators, ts-morph-style tools, anything calling ts.transpileModule or ts.createProgram. Compiling .ts sources is unaffected, because Perry parses TypeScript with SWC.

Removing the native typescript binding (owner decision, 2026-11-04) makes this the default path for every such program.

Why

npm ships tsc as one bundled JavaScript file, node_modules/typescript/lib/typescript.js (9.07 MB; the whole probe program is just 3 modules). Perry compiles a module as a whole. A program that calls only transpileModule still gets native code generated for the entire compiler: scanner, parser, binder, checker, all emitters and the language service. Compile logs show typescript.js lowered in 34 units, ~990 MiB of LLVM IR. LLVM codegen of that IR is nearly all the wall time.

Link-time dead-function elimination (layer 0, commit 1dde1395a4) can drop unreferenced functions from the binary, but only after codegen, so it doesn't reduce compile time. The object cache also didn't hit across different entry programs importing the same typescript.js. Every new program pays the full cost again.

Repro

mkdir ts-import && cd ts-import
cat > package.json <<'JSON'
{
  "name": "ts-import-probe",
  "type": "module",
  "dependencies": { "typescript": "5.8.2" },
  "perry": {
    "compilePackages": ["typescript"],
    "allow": { "compilePackages": ["typescript"] }
  }
}
JSON
npm install
cat > main.ts <<'TS'
import ts from "typescript";
const out = ts.transpileModule("const x: number = 1; export default x;", {
  compilerOptions: { target: ts.ScriptTarget.ES2020, module: ts.ModuleKind.ESNext },
});
console.log(ts.version, JSON.stringify(out.outputText));
TS
node main.ts                                   # reference output, ~0.3 s
/usr/bin/time -l perry compile main.ts -o main # default: facts stage + codegen
PERRY_PROGRAM_FACTS=0 /usr/bin/time -l perry compile main.ts -o main2   # codegen cost alone
./main2                                        # same output as node

On builds that still have the native binding, the compilePackages entry is what forces the real source. Once the binding is removed, a plain import ts from "typescript" takes this path.

Measurements

Perry ced22982 (0.5.1663), M1 mac mini (8 GB, shared host), Node 26.5.1.

program compile wall (facts off) peak RSS binary
transpileModule (ESNext/CJS/TSX, diagnostics) 28.7 min 7.4 GB 78.1 MB
createSourceFile + AST walk 16.9 min 9.4 GB n/a
createProgram + checker + printer + transform 17.0 min 10.3 GB 78.1 MB
for comparison: old SWC-based native binding ~5 s n/a 9.7 MB

Facts stage (PERRY_FACTS_ONLY=1 PERRY_FACTS_REPORT=1): 1061 s wall, 4.1 GB peak, ~12.5M nodes (#13974).

Runtime is fine. All five probes (transpileModule, AST, in-memory and on-disk createProgram with diagnostics and emit, printer/transform) match Node exactly. 600 back-to-back transpileModule calls take 1010 ms in Perry vs 350 ms in Node, at 152 MB vs 382 MB RSS. Short programs start faster than in Node.

What would fix it (ideas, not decided)

  • Reachability before codegen: don't generate code for functions in a module that nothing reachable from the entry references. The link-time DCE already computes "nothing live references this"; doing it before LLVM would cut most of tsc for a transpileModule caller. It's hard for tsc because of its giant closure-scoped namespace objects, so this needs measuring.
  • A content-keyed object cache for package modules: typescript.js compiled once per Perry version and options, then reused across entry programs.
  • Cheaper codegen for huge cold modules: lower optimization level, or more and smaller units with parallel codegen, above a size threshold.
  • #13974: make the facts stage scale, or bail out on modules over a budget.

Found while probing removal of the native typescript binding. Probe sources and logs are in ~/pkgprobe/typescript on the bench mini.

Lingua principale
Rust
Stelle
4.9k
Fork
166
Merge medio
3h 46m
PR unite (30g)
891

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PerryTS/perry

Tutte le issue di PerryTS/perry

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.