ARM64: threads stuck forever on a store instruction with hv_balloon free page reporting (rewriting the same byte via /proc/PID/mem unsticks them)
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 25/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- linux
- Área
- backend, operating-systems
Línea de trabajo
Start by examining the ARM64 WSL2 guest kernel paths implicated in the report: hv_balloon free page reporting, user page faults, and the kernel-side do_swap_page trace. Compare behavior with balloon reporting disabled and collect the offered per-incident traces, especially host-side Hyper-V ETW during a stuck store. Done means identifying and fixing the cause so guest stores complete or fault and recover under memory pressure.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Windows Version
10.0.26300.9818 (Snapdragon X2 Elite, ARM64, 48 GB RAM)
WSL Version
3.0.5.0 (also seen on 2.7.11 before upgrading)
Are you using WSL 1 or WSL 2?
- WSL 2
Kernel Version
6.18.58.1-microsoft-standard-WSL2 (aarch64, PREEMPT_DYNAMIC)
Distro Version
Ubuntu 26.04 LTS
Other Software
Docker Desktop (engine 29.9.2, WSL integration), Node.js tools (Claude Code, MCP servers), PostgreSQL/MariaDB/PHP containers.
.wslconfig:
[wsl2]
memory=20GB
processors=12
swap=8GB
kernelCommandLine=hugepages=1300 preempt=full
[experimental]
autoMemoryReclaim=gradual
sparseVhd=true
Repro Steps
No deterministic repro. It happens several times a day on a busy dev VM (Docker containers, many Node processes, memory pressure that makes the balloon / free page reporting active):
- ARM64 Windows host, WSL 2 with
hv_balloonloaded (default) and free page reporting active (autoMemoryReclaim=gradual, or the defaultpage_reporting_order). - Run a normal multi-process workload for a few hours (Docker containers, Node, Postgres).
- Sooner or later a thread sits at 100% CPU forever on a single store instruction.
Expected Behavior
A user-space store to a mapped, writable page completes, or faults and gets resolved by the guest kernel.
Actual Behavior
A user-space thread gets stuck forever on one store instruction.
-
The thread spins at 100% user time with zero voluntary context switches, and its PC never changes. We sampled it with gdb: two reads 2 s apart, over minutes to hours.
-
The instruction is always a write to memory:
When (UTC-3) Process Instruction at stuck PC 2026-10-31 postgres backends, soketi str/stp2026-11-04 14:03 node (JIT worker) str q3, [x0]insidememcpy, 10 min2026-11-04 (afternoon) node (hooks thread) dc zvainsidememset, ~2 h2026-11-04 (afternoon) codex (Rust, tokio worker) ldadd2026-11-04 16:54 node str q3, [x0]2026-11-04 18:39 node (MCP server) str x3, [x2]2026-11-04 18:41 same process, V8 worker cas x0, x1, [x2]2026-11-04 18:42 another node stur w2, [x1, #19] -
The page is never resolved for that thread. If we write the same byte that is already at the target address through
/proc/PID/mem, the kernel resolves the page from outside (access_remote_vm). The stuck thread then continues in under 3 s: state goes fromRtoSand voluntary switches start to count again. We now do this automatically and it has worked in every case above:python3 -I -c "f=open('/proc/$PID/mem','r+b',buffering=0); f.seek($ADDR); b=f.read(1); f.seek($ADDR); f.write(b)"To us this points to a stale or lost stage-2 mapping or a permission state in the hypervisor for a guest page. The guest believes the page is present and writable, so it never faults it in again. A host-side write path re-establishes it. In our history the problem correlates with
hv_balloonfree page reporting (pages reported to the host and later reused by the guest):- With
initcall_blacklist=init_balloon_drv(balloon disabled, 2026-11-01 18:29 to 2026-11-03 10:51, ~40 h) we saw no stuck store. The only hot threads then were ordinary user-space loops with a changing PC. With the balloon back on, the first kernel-side case came 11.5 h later. - With
page_reporting.page_reporting_order=10it still happened. - Upgrading from WSL 2.7.11 to 3.0.5 did not change it.
- With
Kernel-side variant. The same thing can happen to a store done by the kernel on behalf of a task. On 2026-11-03 22:23:
- A task's
stimegrew by ~100 ticks per second and itsutimestayed constant. - The task was a Node V8 worker thread on CPU 4. The
sysrq-lNMI backtrace showeddo_swap_page→folio_add_new_anon_rmap→mod_memcg_lruvec_state+0x18c(insidememcg_rstat_updated, preemption disabled, a per-cpuldadd). - With the default
PREEMPT(none), that CPU never released the migration stopper. Every latersched_setaffinity/ migration (affine_move_task) piled up behind it. - Load reached 458, processes became unkillable (SIGKILL ignored), and only
wsl --shutdownrecovered the VM. - Booting with
preempt=fullcontains it: one CPU is lost instead of the whole VM. The root cause is still there.
Diagnostic Logs
We keep per-incident dossiers (gdb disassembly around the PC, registers, /proc/PID/task/*/stat|status, sysrq-l/sysrq-w traces, /proc/interrupts, free, cmdline) and can attach them or run collect-wsl-logs.ps1 during the next occurrence if that helps. Please tell us which traces would be most useful, for example Hyper-V ETW providers on the host while a thread is stuck.
Workaround for others hitting this on ARM64: either disable the balloon (kernelCommandLine=initcall_blacklist=init_balloon_drv, at the cost of memory never returning to Windows), or detect threads stuck on a store and rewrite the same byte through /proc/PID/mem as above, plus preempt=full so a kernel-side occurrence doesn't freeze the whole VM.
- Lenguaje dominante
- C++
- Estrellas
- 34.4k
- Forks
- 1.9k
- Merge medio
- 1 d 23 h
- PR fusionados (30 d)
- 166
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/WSL
-
feature wsl2
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
microsoft/WSL#42306 · 3 comentarios ·
Los mantenedores suelen responder en 1 día
-
feature
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
microsoft/WSL#42114 · 3 comentarios ·
Los mantenedores suelen responder en 1 día
-
feature kernel wsl2
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
microsoft/WSL#41107 · 3 comentarios · 1 reacción ·
Los mantenedores suelen responder en 1 día
-
feature kernel
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
microsoft/WSL#41054 · 9 comentarios · 1 reacción ·
Los mantenedores suelen responder en 1 día
-
feature GPU
Dificultad 1/5 1-3 horas Aptitud para principiantes 70/100
microsoft/WSL#13592 · 4 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de microsoft/WSL
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
HarperFast/rocksdb-js#1131 ·
Los mantenedores suelen responder en 1 día
-
area/daemon bug good first issue platform/linux priority/p2
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
ToshiNamura/Deimos#5 ·
-
needs-format
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Niko1221/Strata#6251 · 1 comentario ·
Los mantenedores suelen responder en 5 días
-
[Bug]: NP 管理器文件编辑器里删除键无效Abiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
metasequoiaime/msime#11666 ·
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
patrick-hudson/put.gd-desktop#1679 · 1 comentario ·
Los mantenedores suelen responder en 1 día