Async Pack still deadlocks on hardlinked files in v7.5.22 (#458 not fixed); create({file}) turns it into a silent exit-0 no-op

issue にある決定論的な hardlink ハングを再現し、その後 src/pack.ts の [PROCESS]、[PROCESSJOB]、[ONSTAT]、[ENTRY]、[JOBDONE] 周辺を読んで、キューとジョブスロットの進行を追跡します。src/create.ts で未解決の file-mode promise を確認し、npm test を実行します。hardlink の再現で非同期パッキングが完了し、create({file}) が完全なアーカイブで解決され、この失敗をカバーする回帰テストがあることが完了の条件です。

Summary

The hardlink deadlock reported in #458 is not fixed. It reproduces on v7.5.22 (current latest) on both Linux and Windows, using @jakebailey's original repro from that issue, unmodified.

#458 was closed as fixed in 7.5.15 (7aef486), but the reporter replied on 2026-05-11 that it still hung ("it seems like the jobs count needs a decrement somewhere") and the thread went quiet. Filing fresh since that issue is closed and easy to miss. That instinct was right, and this issue pins down exactly where.

There is also a more dangerous variant: in async file mode (create({file, ...})), the same deadlock produces a silent no-op. The create() promise never settles, all fds close, the event loop drains, and Node exits code 0 — so await tar.create(...) returns nothing, throws nothing, and leaves a truncated archive on disk. We hit this in production; a build pipeline reported success while writing a partial tarball.

Reproduction 1 — @jakebailey's repro from #458, verbatim, on 7.5.22

const { mkdirSync, writeFileSync, linkSync, rmSync } = require('fs');
const { join } = require('path');
const { Pack } = require('tar');

const dir = join(__dirname, 'testdir');
rmSync(dir, { recursive: true, force: true });
mkdirSync(dir);

for (let i = 0; i < 20; i++) {
  const sub = join(dir, `dir${String(i).padStart(3, '0')}`);
  mkdirSync(sub);
  writeFileSync(join(sub, 'file.txt'), `content-${i}`);
}

const src = join(dir, 'dir000', 'hardlink-source.txt');
writeFileSync(src, 'I am the link target');
for (let i = 1; i < 15; i++) {
  linkSync(src, join(dir, `dir${String(i).padStart(3, '0')}`, 'hardlink.txt'));
}

const packer = new Pack({ cwd: __dirname });
packer.add('testdir');
packer.end();

let bytes = 0;
packer.on('data', (chunk) => bytes += chunk.length);
packer.on('end', () => {
  console.log(`OK - Pack finished (${bytes} bytes)`);
  process.exit(0);
});

setTimeout(() => {
  console.log(`HANG - Pack stuck after 5s (read ${bytes} bytes, stream not ended)`);
  process.exit(1);
}, 5000);

tar 7.5.22, node v24.18.1, linux (docker node:24)
HANG - Pack stuck after 5s (read 12800 bytes, stream not ended)

tar 7.5.22, node v24.14.1, win32
HANG - Pack stuck after 5s (read 12800 bytes, stream not ended)

20 directories, 35 files. Deterministic on both platforms.

Reproduction 2 — the silent create({file}) no-op

Trigger: a directory of files hardlinked to siblings outside the set being archived — the normal shape of a hardlink-cloned build tree (cp -al, clone-by-link, dedupe tooling). No entry with a matching inode is ever added to the archive, so the pending-link park has nothing to wait for.

import { mkdirSync, writeFileSync, linkSync, rmSync, statSync } from 'node:fs'
import { join } from 'node:path'
import * as tar from 'tar'

const N = 3000
const root = join(process.cwd(), 'tree')
const out = join(process.cwd(), 'out.tgz')

rmSync(root, { recursive: true, force: true })
rmSync(out, { force: true })
mkdirSync(join(root, 'src'), { recursive: true })
mkdirSync(join(root, 'elsewhere'), { recursive: true })

const files = []
for (let i = 0; i < N; i++) {
  const name = `f${String(i).padStart(5, '0')}.bin`
  // the only other link to this inode lives outside the archived set
  const outside = join(root, 'elsewhere', name)
  writeFileSync(outside, Buffer.alloc(512, i % 251))
  linkSync(outside, join(root, 'src', name))
  files.push(`src/${name}`)
  // interleaved plain nlink=1 files: these are what pin the JOBS slots
  if (i % 5 === 0) {
    const plain = `p${String(i).padStart(5, '0')}.bin`
    writeFileSync(join(root, 'src', plain), Buffer.alloc(256, 7))
    files.push(`src/${plain}`)
  }
}

let settled = 'no'
tar
  .create(
    { file: out, cwd: root, gzip: { level: 1 }, portable: true, noDirRecurse: true, strict: true },
    files,
  )
  .then(
    () => (settled = 'resolved'),
    e => (settled = `rejected (${e.message})`),
  )

process.on('exit', code => {
  console.log(
    `SETTLED: ${settled}\nexit code: ${code}\noutput bytes: ${statSync(out).size}`,
  )
})

tar 7.5.22, node v24.14.1, win32 -- 3600 entries, nlink=2

SETTLED: no
exit code: 0
output bytes: 10

5/5 runs. Nothing thrown, nothing logged, exit 0, and a 10-byte truncated archive containing zero of the 3600 entries. Variants of this script that wedge later leave a partial archive instead (one run wrote 1977 of 3600 entries before stranding) — either way the caller cannot tell.

This one is interleave-sensitive: it did not trigger on Linux at 3k/10k/30k files in a container, whereas Reproduction 1 is deterministic on both platforms. Reproduction 1 is the one to debug against; Reproduction 2 is included because the silent success failure mode is what makes this expensive in practice.

The deadlock state

Instrumenting Pack and dumping its internals at the moment the event loop drains (Reproduction 2, win32, unmodified v7.5.22):

this[JOBS]        = 4
this.jobs (limit) = 4
this[ENDED]       = true
this[PROCESSING]  = false
queue length      = 3269
pendingLinks size = 8
queue (first 8):
  [0] src/p00275.bin pending=false pendingLink=false entry=true  piped=false nlink=1
  [1] src/f00276.bin pending=false pendingLink=true  entry=true  piped=false nlink=2
  [2] src/f00277.bin pending=true  pendingLink=true  entry=false piped=false nlink=2
  [3] src/f00278.bin pending=true  pendingLink=true  entry=false piped=false nlink=2
  [4] src/f00279.bin pending=true  pendingLink=true  entry=false piped=false nlink=2
  [5] src/f00280.bin pending=true  pendingLink=true  entry=false piped=false nlink=2
  [6] src/p00280.bin pending=false pendingLink=false entry=true  piped=false nlink=1
  [7] src/f00281.bin pending=true  pendingLink=true  entry=false piped=false nlink=2
  ... (queue jobs holding an unpiped entry: 4)

this[JOBS] === this.jobs, everything added, 3269 entries never written, and all four occupied job slots are held by WriteEntrys that were created but never piped.

Note the head here is p00275.bin, a plain nlink=1 file whose entry exists but was never piped. The head does not have to be a hardlink — hardlinks are what create the state, not what the head is stuck on.

Root cause

The job-slot accounting and the queue-head rule are mutually deadlocking:

1. Only the queue head is ever piped. Both [PIPE] call sites are gated on job === this[CURRENT] (src/pack.ts:393, src/pack.ts:435).
2. A slot is only released by piping. [ENTRY] takes a slot (src/pack.ts:460) and returns it only in [JOBDONE] (src/pack.ts:364), which is wired to the entry's 'end' (src/pack.ts:464). An unpiped WriteEntry buffers and never ends, so read-ahead entries hold their slots indefinitely.
3. But the head itself is gated by the slot limit. [PROCESS]'s loop starts at the head and is bounded by !!w && this[JOBS] < this.jobs (src/pack.ts:335). Once JOBS === jobs the loop body never runs, so [PROCESSJOB] is never called on the head — and in steady state [PROCESS] is the only thing that would pipe it.

So when the head needs one more [PROCESSJOB] call to get piped, while all slots are held by read-ahead entries that can only be released by becoming the head, nothing can move. [JOBDONE] is the only other path that calls [PROCESSJOB], and it can't fire, because firing requires an entry to end.

Hardlinks are what push the pack into that state. [ONSTAT] parks any nlink > 1 file that isn't the current head (src/pack.ts:284-300), deferring it until an in-archive entry with the same dev:ino completes. A parked job stays in the queue but consumes no slot, so [PROCESS]'s walk steps over runs of them and creates entries for plain files far ahead of the head — each taking a slot, none piped. When the links point outside the archived set, the sibling never arrives.

A narrower fix does not work. My first attempt unparked a pending && pendingLink head inside [PROCESS] regardless of JOBS. It still deadlocked, because the head is often not parked at all by then: [JOBDONE]'s unpark loop (src/pack.ts:373-374) sets job.pending = false and calls [PROCESSJOB] while the job is not current, which creates its entry (taking a slot) without piping it, and leaves pendingLink set — row [1] above. And the head can be a plain file that simply never got piped — row [0]. Any fix keyed on the pending-link flags misses both.

sync is immune because the parking branch is explicitly !this.sync (src/pack.ts:286) — which is also the practical workaround today.

The silent-exit behaviour comes from createFile (src/create.ts:30-33): the returned promise only resolves on the write stream's 'close' and rejects on stream/pack 'error'. A deadlocked pack emits neither, so the promise is abandoned and the process exits 0.

Suggested fix

Treat jobs as a read-ahead limit, not a head limit. The head must always be allowed to make progress, since it is the only job that can release a slot:

--- a/src/pack.ts
+++ b/src/pack.ts
@@ [PROCESS]()
     this[PROCESSING] = true
+    // The head is the only job that is ever piped, and piping is the only
+    // thing that frees a job slot, so the head must never be gated by the
+    // job limit -- otherwise a full job table deadlocks the whole pack.
+    const head = this[CURRENT]
+    if (head) {
+      this[PROCESSJOB](head)
+    }
     for (
       let w = this[QUEUE].head;
       !!w && this[JOBS] < this.jobs;
       w = w.next
     ) {

Results on a clean v7.5.22 checkout with only that change (rebuilt via npm run prepare, patch confirmed present in dist/esm/pack.js):

The 2 failures are pre-existing on unmodified v7.5.22 in the same container (test/unpack.js → "ignore self-referential hardlinks", async + sync snapshot); the patch changes neither the count nor the names.

I'm not attached to this particular patch — it's the minimal change that satisfies the invariant, and there may be a tidier place to enforce "the head is exempt from the cap". Happy to open a PR with a regression test if that's useful.

The other obvious direction — verifying at park time that the link target is actually in the pending set — doesn't look workable, since entries stream in via add()/write() and the full set isn't known when [ONSTAT] has to decide.

Environment

• tar 7.5.22 (the parking branch and the JOBS < this.jobs gate are also present unchanged in the shipped dist/esm/index.min.js bundle, not just the readable sources)
• node v24.18.1 on linux (node:24 docker) and v24.14.1 on win32 (Windows 11)
• Originally hit on a ~13 GB hardlink-cloned build tree, wedging deterministically on a run of ~1.7k small nlink=3 PNGs

cc @jakebailey — follow-up to your #458 comment; "jobs count needs a decrement somewhere" was pointing at exactly this.