collect_packages_in_node_modules walks symlinked node_modules with no visited set: 3 fully-connected workspace packages hang perry compile at Collecting modules (100% CPU, no output)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- rust
- Domain
- build-system
Research direction
Start in crates/perry/src/commands/compile/resolve.rs, reading collect_packages_in_node_modules and the root enumeration around lines 320–397; trace its caller from host_config.rs:899. Reproduce with the provided three-package symlink cycle, then run the relevant compiler tests or add a regression test for terminating enumeration. Done means package collection terminates for cyclic symlinked node_modules and the compile proceeds.
Written by the indexing model from the issue text.
Description
What happened
A perry compile <file> never leaves Collecting modules... on a pnpm workspace where workspace packages are symlinked into each other's node_modules. The real compiler binary spins at 98-100% CPU indefinitely with flat RSS, and the Compile package wildcard: expanded to N installed package(s) line never prints.
The entry file has no imports at all — it is a one-line console.log. Against the same workspace, moving a single vendor/<pkg>/node_modules directory aside makes the identical command finish in ~3 s.
What you expected
Package enumeration terminates and the compile finishes in a few seconds.
Minimal reproduction
The trigger is the combination of two things: an ancestor node_modules (so ancestor_node_modules_dirs starts the walk) and a symlink cycle whose nodes have more than one outlet each.
mkdir -p repro/node_modules/@deepseek-ai repro/vendor
echo "console.log(1)" > repro/main.ts
for i in 0 1 2; do
mkdir -p repro/vendor/p$i/node_modules/@deepseek-ai
echo "{\"name\":\"@deepseek-ai/p$i\",\"version\":\"1.0.0\",\"main\":\"index.js\"}" > repro/vendor/p$i/package.json
echo "export const v = 1" > repro/vendor/p$i/index.js
ln -s ../../vendor/p$i repro/node_modules/@deepseek-ai/p$i
done
for i in 0 1 2; do for j in 0 1 2; do
[ "$i" != "$j" ] && ln -s ../../../p$j repro/vendor/p$i/node_modules/@deepseek-ai/p$j
done; done
cd repro && perry compile main.ts -o app
Measured results (same machine, same command):
| graph | result |
|---|---|
N=2, one outlet each (p0 -> p1 -> p0) |
completes in ~3 s, 14.2 MB binary |
| N=3, each package links to the other two | hangs >15 s, 98-100% CPU, no output |
| N=8, fully connected | hangs >42 s, no output |
| real workspace (350+ packages, one cyclic group of 8 vendored packages) | hangs >60 s, no output |
On the real workspace, moving any single vendor/<pkg>/node_modules aside (cordis, include, loader, group, logger-console, hmr, timer or schemastery) turns the hang into a 3 s success. Removing the nested directories for just timer is enough.
Entry point matters: copying the same cyclic tree to a directory whose ancestors have no node_modules does not reproduce — the scan never starts there. My first attempts failed exactly that way.
Root cause (source read, main @ 0.5.1520)
crates/perry/src/commands/compile/resolve.rs, collect_packages_in_node_modules (~lines 351-397):
out.entry(name.to_string()).or_default().push(path.clone());
let nested = path.join("node_modules");
if nested.is_dir() {
collect_packages_in_node_modules(&nested, out);
}
The recursion has no visited set and no depth bound, and path keeps the symlinked spelling (never canonicalized), so a cycle re-enters the same physical directory forever. enumerate_installed_package_roots canonicalizes and dedupes roots at lines 320-326, but only after the walk returns — which never happens.
The walk is entered from host_config.rs:899 -> enumerate_installed_package_roots(project_root) -> ancestor_node_modules_dirs(project_root).
A single-outlet cycle (p0 -> p1 -> p0, one link each) terminates only because the accumulating path exceeds the platform path limit (ENAMETOOLONG) after ~17 levels; each added outlet multiplies the number of paths that fit, so 3 fully-connected nodes are already enough to hang.
Environment
- Perry version: 0.5.1520 (
perry --version) - Host OS: macOS 27.0.1 (build 26A434), arm64
- Target: native
- Installed via: npm package
@perryts/perry(installed with bun)
Diagnostic output
$ perry compile main.ts -o app
Collecting modules...
<nothing further for >60 s>
sample on the spinning process shows a single thread inside perry-main (binary is stripped, so frames are ???). ps shows the wrapper node .../perry at 0% CPU / 55 MB while the child @perryts/perry-darwin-arm64/bin/perry holds 99-100% CPU. Note that killing the wrapper leaves that child orphaned at 100% CPU — it needs a separate kill.
Anything else
Workaround I use locally: move vendor/*/node_modules aside for the duration of the compile, then restore (guarded by a trap + manifest so an interrupted run can recover).
Suggested fix: thread a visited: HashSet<PathBuf> of canonicalized directory paths through collect_packages_in_node_modules (and collect_packages_in_bun_store) and return early on a repeat. That also removes the duplicate roots the post-pass at 320-326 currently has to dedupe.
It would also help if this class of failure were bounded and visible rather than silent: a depth or visited-node cap plus a progress line would turn a 100%-CPU hang into a diagnosable message.
- Dominant language
- Rust
- Stars
- 4.9k
- Forks
- 166
- Avg merge
- 3h 23m
- Merged PRs (30d)
- 911
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from PerryTS/perry
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 67/100
Maintainers usually reply within 1 day
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 56/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
yologdev/yoyo-evolve#1053 ·
-
bug lompi security sev-9
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
area:config area:security bug good first issue platform:linux platform:windows priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
niravpatidar37/mcpsum#80 ·
Maintainers usually reply within 1 day