ARM64: threads stuck forever on a store instruction with hv_balloon free page reporting (rewriting the same byte via /proc/PID/mem unsticks them)
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 25/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- linux
調査の方向性
Start by examining the ARM64 WSL2 guest kernel paths implicated in the report: hv_balloon free page reporting, user page faults, and the kernel-side do_swap_page trace. Compare behavior with balloon reporting disabled and collect the offered per-incident traces, especially host-side Hyper-V ETW during a stuck store. Done means identifying and fixing the cause so guest stores complete or fault and recover under memory pressure.
索引モデルが issue の本文から書いたものです。
説明
Windows Version
10.0.26300.9818 (Snapdragon X2 Elite, ARM64, 48 GB RAM)
WSL Version
3.0.5.0 (also seen on 2.7.11 before upgrading)
Are you using WSL 1 or WSL 2?
- WSL 2
Kernel Version
6.18.58.1-microsoft-standard-WSL2 (aarch64, PREEMPT_DYNAMIC)
Distro Version
Ubuntu 26.04 LTS
Other Software
Docker Desktop (engine 29.9.2, WSL integration), Node.js tools (Claude Code, MCP servers), PostgreSQL/MariaDB/PHP containers.
.wslconfig:
[wsl2]
memory=20GB
processors=12
swap=8GB
kernelCommandLine=hugepages=1300 preempt=full
[experimental]
autoMemoryReclaim=gradual
sparseVhd=true
Repro Steps
No deterministic repro. It happens several times a day on a busy dev VM (Docker containers, many Node processes, memory pressure that makes the balloon / free page reporting active):
- ARM64 Windows host, WSL 2 with
hv_balloonloaded (default) and free page reporting active (autoMemoryReclaim=gradual, or the defaultpage_reporting_order). - Run a normal multi-process workload for a few hours (Docker containers, Node, Postgres).
- Sooner or later a thread sits at 100% CPU forever on a single store instruction.
Expected Behavior
A user-space store to a mapped, writable page completes, or faults and gets resolved by the guest kernel.
Actual Behavior
A user-space thread gets stuck forever on one store instruction.
-
The thread spins at 100% user time with zero voluntary context switches, and its PC never changes. We sampled it with gdb: two reads 2 s apart, over minutes to hours.
-
The instruction is always a write to memory:
When (UTC-3) Process Instruction at stuck PC 2026-10-31 postgres backends, soketi str/stp2026-11-04 14:03 node (JIT worker) str q3, [x0]insidememcpy, 10 min2026-11-04 (afternoon) node (hooks thread) dc zvainsidememset, ~2 h2026-11-04 (afternoon) codex (Rust, tokio worker) ldadd2026-11-04 16:54 node str q3, [x0]2026-11-04 18:39 node (MCP server) str x3, [x2]2026-11-04 18:41 same process, V8 worker cas x0, x1, [x2]2026-11-04 18:42 another node stur w2, [x1, #19] -
The page is never resolved for that thread. If we write the same byte that is already at the target address through
/proc/PID/mem, the kernel resolves the page from outside (access_remote_vm). The stuck thread then continues in under 3 s: state goes fromRtoSand voluntary switches start to count again. We now do this automatically and it has worked in every case above:python3 -I -c "f=open('/proc/$PID/mem','r+b',buffering=0); f.seek($ADDR); b=f.read(1); f.seek($ADDR); f.write(b)"To us this points to a stale or lost stage-2 mapping or a permission state in the hypervisor for a guest page. The guest believes the page is present and writable, so it never faults it in again. A host-side write path re-establishes it. In our history the problem correlates with
hv_balloonfree page reporting (pages reported to the host and later reused by the guest):- With
initcall_blacklist=init_balloon_drv(balloon disabled, 2026-11-01 18:29 to 2026-11-03 10:51, ~40 h) we saw no stuck store. The only hot threads then were ordinary user-space loops with a changing PC. With the balloon back on, the first kernel-side case came 11.5 h later. - With
page_reporting.page_reporting_order=10it still happened. - Upgrading from WSL 2.7.11 to 3.0.5 did not change it.
- With
Kernel-side variant. The same thing can happen to a store done by the kernel on behalf of a task. On 2026-11-03 22:23:
- A task's
stimegrew by ~100 ticks per second and itsutimestayed constant. - The task was a Node V8 worker thread on CPU 4. The
sysrq-lNMI backtrace showeddo_swap_page→folio_add_new_anon_rmap→mod_memcg_lruvec_state+0x18c(insidememcg_rstat_updated, preemption disabled, a per-cpuldadd). - With the default
PREEMPT(none), that CPU never released the migration stopper. Every latersched_setaffinity/ migration (affine_move_task) piled up behind it. - Load reached 458, processes became unkillable (SIGKILL ignored), and only
wsl --shutdownrecovered the VM. - Booting with
preempt=fullcontains it: one CPU is lost instead of the whole VM. The root cause is still there.
Diagnostic Logs
We keep per-incident dossiers (gdb disassembly around the PC, registers, /proc/PID/task/*/stat|status, sysrq-l/sysrq-w traces, /proc/interrupts, free, cmdline) and can attach them or run collect-wsl-logs.ps1 during the next occurrence if that helps. Please tell us which traces would be most useful, for example Hyper-V ETW providers on the host while a thread is stuck.
Workaround for others hitting this on ARM64: either disable the balloon (kernelCommandLine=initcall_blacklist=init_balloon_drv, at the cost of memory never returning to Windows), or detect threads stuck on a store and rewrite the same byte through /proc/PID/mem as above, plus preempt=full so a kernel-side occurrence doesn't freeze the whole VM.
- 主要言語
- C++
- スター
- 34.4k
- フォーク
- 1.9k
- 平均マージ
- 1日 23時間
- マージ済み PR(30日)
- 166
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
microsoft/WSL のほかの issue
-
feature wsl2
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100
microsoft/WSL#42306 · コメント 3 件 ·
メンテナーはふだん 1 日以内に返信
-
feature
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
microsoft/WSL#42114 · コメント 3 件 ·
メンテナーはふだん 1 日以内に返信
-
feature kernel wsl2
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
microsoft/WSL#41107 · コメント 3 件 · リアクション 1 件 ·
メンテナーはふだん 1 日以内に返信
-
feature kernel
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
microsoft/WSL#41054 · コメント 9 件 · リアクション 1 件 ·
メンテナーはふだん 1 日以内に返信
-
feature GPU
難易度 1/5 1〜3時間 初心者へのやさしさ 70/100
microsoft/WSL#13592 · コメント 4 件 ·
メンテナーはふだん 1 日以内に返信
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
HarperFast/rocksdb-js#1131 ·
メンテナーはふだん 1 日以内に返信
-
area/daemon bug good first issue platform/linux priority/p2
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
ToshiNamura/Deimos#5 ·
-
needs-format
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
Niko1221/Strata#6251 · コメント 1 件 ·
メンテナーはふだん 5 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
metasequoiaime/msime#11666 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
patrick-hudson/put.gd-desktop#1679 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信