Detached Harper children are orphaned permanently when the runner dies by SIGKILL/SIGHUP — reap guard only covers exit/SIGINT/SIGTERM
@kriszyp đang làm issue này rồi.
Từ ngày 21/8/2026.
Đánh giá
Issue này chưa được đánh giá.
Mô tả
Summary
trackHarperProcess reaps spawned Harper children on exit, SIGINT, and SIGTERM. It does not cover the ways a test runner most often actually dies — SIGKILL, SIGHUP, or an abrupt crash — and because Harper is spawned detached (its own process group) the child does not die with its parent. The result is a Harper instance that survives indefinitely with PPID 1, still holding its listening ports.
Evidence (observed on a dev machine, 2026-08-20)
Nine orphaned dist/bin/harper.js processes, all PPID 1, all cwd'd in one harper-pro worktree, ages 16–18 hours:
PID PPID %CPU ELAPSED
93582 1 99.6 18:15:17
94348 1 99.6 18:13:33
29069 1 98.7 16:39:01
28383 1 98.7 16:40:41
28458 1 98.5 16:40:26
2574 1 98.4 18:00:40
93587 1 98.1 18:15:13
3839 1 97.9 17:59:05
2577 1 96.3 18:00:35
Combined 879% CPU (~8.8 cores). Machine load average was 21.8 with 0.14% idle; killing these nine plus two unrelated runaway jobs took it to 6.9 / 91% idle.
They were still holding their ports, which is the part that breaks subsequent runs:
node 93582 ... TCP 127.0.0.7:8883 (LISTEN)
node 93582 ... TCP 127.0.0.7:1883 (LISTEN)
node 93582 ... TCP 127.0.0.7:9933 (LISTEN)
node 2574 ... TCP 127.0.0.10:9933 (LISTEN)
node 28383 ... TCP 127.0.0.15:9933 (LISTEN)
A separate 18-day-old orphaned mocha runner was also present on the same box, so this is not a one-off.
All nine exited cleanly on a plain SIGTERM — nothing was wedged at the OS level; no one had signalled them.
Why the current guard misses
src/harperLifecycle.ts (trackHarperProcess, ~L704 on main):
process.once('exit', reapAll);
// SIGINT/SIGTERM don't fire 'exit'; reap, then re-raise so the runner still terminates normally.
for (const signal of ['SIGINT', 'SIGTERM'] as const) {
process.once(signal, () => {
reapAll();
process.kill(process.pid, signal);
});
}
Not covered:
| Runner death | Handler runs? | Why it matters |
|---|---|---|
SIGKILL |
No — uncatchable | CI job cancellation, OOM killer, kill -9 |
SIGHUP |
No — not registered | terminal/session teardown |
hard crash / SIGABRT |
No | runner segfault or abort |
SIGINT / SIGTERM / normal exit |
Yes | already handled |
The detached: true spawn is correct and deliberate (it's what lets teardown signal the whole group), but it also means the child is insulated from anything that kills the runner's group. Nothing in the child notices its parent is gone.
CI cancellation is the case worth calling out: cancelled jobs are a routine occurrence in the harper-pro stress matrix, and cancellation lands as SIGKILL.
The invariant
"A Harper child never outlives its runner" is currently maintained by enumerating parent-side signals — so every uncatchable or unregistered death is a hole. It would be more robust to enforce it from the child side, where no parent cooperation is needed.
Suggested direction (not prescriptive)
- Parent-liveness watchdog in the child — the durable fix. Have the child poll for
getppid() === 1(or watch the parent via a pipe that closes on parent death) and self-terminate. SurvivesSIGKILLbecause it needs nothing from the parent. - Register
SIGHUPalongsideSIGINT/SIGTERM— cheap, closes one hole, doesn't help withSIGKILL. - Startup sweep — on
startHarper, reap stale Harper processes whose parent is gone before claiming an address. Complements #13, which is the same runner-killed-mid-shard root cause applied to the loopback address pool.
(1) is the only one that closes the SIGKILL case; (3) is the pragmatic backstop and would also reduce the port-contention failures behind #8 and #28.
Related
- #13 — Loopback pool leaks addresses from runners killed mid-shard (dead-PID sweep only runs when the pool is full) — same root cause (runner killed mid-shard), different leaked resource.
- #8 — startHarper readiness deadline and teardown loopback recycle cause intermittent failures under sharded CI and #28 — Teardown's port-release wait probes all five fixed Harper ports unconditionally — orphans holding ports are one source of the contention these describe.
Separately: the reason these orphans were hot rather than idle is a Harper-side unbounded spin lock, filed separately against harper. Reaping and the spin are independent defects that compounded here.
Version
@harperfast/integration-testing 0.7.1; guard is unchanged on main.
- Ngôn ngữ chính
- TypeScript
- Star
- 1
- Fork
- 0
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Chuẩn bị môi trường
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của HarperFast/integration-testing
-
setupHarperWithFixture overwrites ctx.harper, dropping pre-set hostname (breaks multi-node add_node)Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
-
enhancement good first issue
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
-
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 74/100
-
A listener on all interfaces silently receives a test node's connections on macOS; the conflict canary cannot see itCó thể đã có người làm @dawsontoth đã nhận 1 ngày trước. Đang mở
HarperFast/integration-testing#38 · 1 bình luận · 1 người được giao ·
-
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 58/100
Tất cả issue của HarperFast/integration-testing
Issue tương tự
-
needs:triage
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 84/100
Maintainer thường phản hồi trong vòng 1 ngày
-
ai-discovered
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 83/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 88/100
jessepollak/home#1627 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
agent-canvas bug llm priority:low ready-for-dev
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 82/100
OpenHands/OpenHands#17806 · 3 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
radius-project/ai-extensions#923 ·
Maintainer thường phản hồi trong vòng 1 ngày