[Bug] OpenCode adapter leaks ChildProcess exit listeners until an LXC host freezes: "11 exit listeners added to [ChildProcess]"

未關閉
#39,121 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 1 天內回覆

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
45/100
Issue 類型
缺陷
描述清晰度
基本清楚
活躍度
活躍
技術堆疊
node.js, typescript
領域
backend, devtools

研究方向

Start by locating the OpenCode adapter’s ChildProcess exit listener registration and tracing how recoverShells, event-stream reconnects, and borrowed-session opens reuse child handles. Check whether listeners remain attached after exit or are added again to the same child; the issue does not name source files or tests. Done when repeated reconnect and retry paths do not accumulate exit listeners or retain exited child processes.

由索引模型根據 Issue 內容生成。

描述

needs more info

[Bug] t3code.service leaks ChildProcess exit listeners until the whole LXC freezes: MaxListenersExceededWarning ... 11 exit listeners added to [ChildProcess]

Environment
  • T3 Code Server t3 serve --host 0.0.0.0 --base-dir /opt/t3code, systemd system unit t3code.service, Debian 13 inside an unprivileged Proxmox LXC (6 vCPU, 16 GiB), node v24.22.3 bundled SEA binary.
  • Reproduced on 0.1.10 (npm view t3 dist-tags.latest = 0.1.10, so no stable upgrade exists). A test runtime 0.1.11-nightly.20261119.4928 was installed on 2026-11-19 08:32 EST to see whether it still leaks; results below.
  • Provider: OpenCode adapter (opencode serve children + opencode-pty).
What happened

The container froze completely twice on 2026-11-19 (06:44 and ~07:20 EST) and twice on 2026-11-17 (00:56, 04:16). Nothing self-healed: the guest journal goes silent roughly 20 minutes before a human has to force a stop/start from the Proxmox side (vzstop:4038:root@pam 07:07:45 + vzstart 07:07:52, again 08:07:58 + 08:08:12). Inside the guest the journal simply ends (06:49:04, 07:42:04) with no shutdown lines, and lxc-stop returns in ~3 s because all that RAM vanishes at once.

Numbers (container-scoped node_exporter, 5-min samples):

06:40  MemAvailable 7168 MiB   AnonPages 2337 MiB   Active_anon  967 MiB  load1  4.5
06:45  MemAvailable 2925 MiB   AnonPages 6552 MiB   Active_anon 5581 MiB  load1 13.5  procs_running 21
06:50  MemAvailable  780 MiB                        load1 77.7   procs_blocked 11
07:10  MemAvailable 4743 MiB   (after a forced restart)
07:30  MemAvailable  563 MiB   (growth resumed immediately)

While frozen, node_pressure_io_stalled_seconds_total and node_pressure_memory_stalled_seconds_total repeat identical values (06:44-06:52) - even node_exporter was blocked, which is why the guest stops logging while Proxmox still sees a live PID.

The signature points at ChildProcess exit listeners

The server logs itself, minutes before each cliff:

2026-11-19T05:13:56 t3[411515]: (node:411515) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added to [ChildProcess]. MaxListeners is 10. Use emitter.setMaxListeners() to increase limit
2026-11-19T06:44:00 t3[411515]: (node:411515) MaxListenersExceededWarning: ... same ...
2026-11-19T08:09:16 t3[404]:   (node:404)   MaxListenersExceededWarning: ... same ...

06:44:00 is five minutes before the 06:45 cliff. The same process also logs WARN (#9692): Lost the OpenCode event stream; reconnecting. (5 occurrences over 3 days) and a 54-line storm of

TimeoutError: Operation timed out after '5s'
    at OpenCode2Adapter.recoverShells (file:///root/.t3/runtime/versions/0.1.10/t3:426721:50)
    at OpenCode2Adapter.openSession (...:427739:19)
    at OpenCode2Adapter.borrowAndOpenSession (...:431816:81)
    at ProviderInstallationGate.execute (...:274260:45)
    at orchestrationV2.providerTurnStart.start (...:363638:268)

so listeners appear to be re-added on every event-stream reconnect / recoverShells retry / borrowed-session open, on ChildProcess objects that stay referenced.

Growth is steep and starts from a cold idle start: after the 08:19:55 restart, systemctl show t3code -p MemoryCurrent -p MemoryPeak went 742 MiB -> 1.99 GiB MemoryPeak within 9 minutes, RSS being ~480 MiB in the node main process + ~330 MiB in one opencode child, with no thread doing heavy work. It is the top-RSS process in the container by 3x.

Ruled out on our side (so this is not a workload report)
  • Disk: root 126 GiB with 23 GiB free (82 %).
  • Host: hypervisor up 17 days, no hung_task/ata/I/O-error kernel lines, smartctl PASSED on all three disks, no ZFS.
  • cgroup OOM: /sys/fs/cgroup/lxc/4038/memory.events shows oom 0 oom_kill 0 - the kernel never killed anything, the box just thrashed (5.5 GiB of container swap in use).
  • Coredumps: none. lxcfs: NRestarts=0. A per-unit sampler confirms the growth is inside t3code.service, not in any other service, and it is anonymous heap (not page cache, not /tmp tmpfs, which stayed flat at 7355 MiB available across the cliff).
  • The OpenCode reconnect symptom matches closed #37396; the listener accumulation and the RAM growth do not appear to be covered there, and no existing issue mentions exit listeners or ChildProcess listeners.
Asks
  1. Detach (or use {once:true} for) the exit listeners registered on OpenCode / provider ChildProcess handles when the child exits, and de-duplicate registration per child instead of per reconnect - the warning names the exact emitter, so it should be findable.
  2. Consider a defensive setMaxListeners only as a symptom mask; the leak of the retained ChildProcess + closure is the RAM cost, and 0.1.10 will happily reach the container limit.
  3. Nice to have: log per-adapter retained-process counts (or expose them via a debug endpoint) so operators can attribute growth without a 3-day Prometheus history.
Workaround in place

/etc/systemd/system/t3code.service.d/memory-cap.conf = MemoryHigh=4G, MemoryMax=6G, OOMPolicy=stop, Restart=always, RestartSec=5, so the server is killed and back in ~5 s instead of freezing the whole container until someone presses Restart in the hypervisor UI. A V8 --max-old-space-size cap was rejected because the binary is a node SEA bundle and NODE_OPTIONS inheritance to its opencode children is not reliable.

主要語言
TypeScript
星號
24.8k
分支
6.4k
平均合併
4 小時 29 分鐘
30 天內合併 PR
488

環境準備

在 Codespaces 中開啟

在瀏覽器裡用你自己的 GitHub 帳號啟動這個專案的開發容器。

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

pingdotgg/t3code 的其他 Issue

查看 pingdotgg/t3code 的全部 Issue

相似的 Issue

更多 TypeScript Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。