Mirrored networking: host-only 127.0.0.1 TCP traffic spins all cores in `Fse.sys!FsePortPoolGetPartitionsForPortUncached` (exclusive spin lock), freezing the desktop
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 5/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- cpp
- Ambito
- networking, operating-systems, performance
Direzione di ricerca
Start by building and running the main.go repro in mirrored mode (fserepro.exe load -dur 10s -rounds 10 -idle 4000) with the probe mode running alongside, then capture a wpr -start CPU trace and inspect the ExpWaitForSpinLockExclusiveAndAcquire samples under FsePortPoolGetPartitionsForPortUncached. From there, find the port-pool partition lookup and its cache in the fse.sys sources in this repo and read how reservations are added and removed on connection churn. Done means the uncached path no longer takes the pool-wide exclusive spin lock per loopback packet, or the cache stops going cold under churn, verified by the repro's kernel CPU per connection dropping back to the WSL-stopped baseline.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Windows Version
Microsoft Windows [Version 10.0.26300.9818] (Windows 11 Home, 26H2)
WSL Version
3.0.3.0
Are you using WSL 1, WSL 2, or WSLC?
WSL 2
Kernel Version
6.18.54.1-1
Distro Version
Ubuntu 24.04
Other Software
.wslconfig:[wsl2] networkingMode=mirrored dnsTunneling=true autoProxy=true processors=16- Driver versions:
fse.sys10.0.26100.9818,tcpip.sys10.0.26100.8875,afd.sys10.0.26100.9949 - CPU: AMD Ryzen 9 7950X (16 cores / 32 threads), 96 GB RAM
- Other network-related software is installed (VMware Workstation, Tailscale, a wintun-based TUN client). None of their drivers appear in the hot call stacks below, and the problem goes away with only
wsl --shutdown.
Repro Steps
The traffic in this repro is entirely between Windows processes on the host (127.0.0.1 → 127.0.0.1 inside one Windows process). Nothing is sent to or from the Linux VM. The only variable is whether a mirrored-mode WSL VM is running.
- Set
networkingMode=mirroredand start a distro, keeping the VM up:wsl -d Ubuntu-24.04 -e sleep 600 - On the Windows host, build the program below (Go 1.27):
go build -o fserepro.exe main.go - Run it.
-idle 4000holds 4000 idle loopback connections open, then 32 workers repeatedly connect, do 10 round trips of 4 KB, and close:
In another window,fserepro.exe load -dur 10s -rounds 10 -idle 4000fserepro.exe probe -dur 14smeasures how late a 100 ms sleep wakes up. - Run
wsl --shutdownand repeat step 3.
main.go (standalone, standard library only)
// fserepro drives high-rate TCP traffic over 127.0.0.1 between two endpoints
// in one Windows process, and reports throughput and the user/kernel CPU split.
// The "probe" mode measures how late a 100 ms sleep wakes up, as a proxy for
// system responsiveness while the load runs.
package main
import (
"flag"
"fmt"
"io"
"net"
"os"
"runtime"
"sync"
"sync/atomic"
"syscall"
"time"
)
func cpuTimes() (user, kernel time.Duration) {
var c, e, k, u syscall.Filetime
h, _ := syscall.GetCurrentProcess()
if err := syscall.GetProcessTimes(h, &c, &e, &k, &u); err != nil {
return 0, 0
}
ft := func(f syscall.Filetime) time.Duration {
return time.Duration((int64(f.HighDateTime)<<32 | int64(f.LowDateTime)) * 100)
}
return ft(u), ft(k)
}
func lowerPriority() {
const belowNormal = 0x4000
k := syscall.NewLazyDLL("kernel32.dll")
h, _ := syscall.GetCurrentProcess()
k.NewProc("SetPriorityClass").Call(uintptr(h), belowNormal)
}
func echo(ln net.Listener) {
for {
c, err := ln.Accept()
if err != nil {
return
}
go func() {
defer c.Close()
io.Copy(c, c)
}()
}
}
// load runs workers that each open a connection, do `rounds` request/response
// exchanges of `payload` bytes, and close it. rounds <= 0 keeps one connection
// per worker for the whole run. Before the load starts, `idle` extra
// connections are opened and held without traffic, so that many local ports
// are in use while the workers run.
func load(workers int, dur time.Duration, payload, rounds, idle int) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
fmt.Println("listen:", err)
os.Exit(1)
}
go echo(ln)
addr := ln.Addr().String()
held := make([]net.Conn, 0, idle)
for len(held) < idle {
c, err := net.Dial("tcp", addr)
if err != nil {
fmt.Printf("idle dial stopped at %d: %v\n", len(held), err)
break
}
held = append(held, c)
}
var trips, conns, errs atomic.Int64
deadline := time.Now().Add(dur)
u0, k0 := cpuTimes()
start := time.Now()
var wg sync.WaitGroup
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
out, in := make([]byte, payload), make([]byte, payload)
for time.Now().Before(deadline) {
c, err := net.Dial("tcp", addr)
if err != nil {
errs.Add(1)
time.Sleep(10 * time.Millisecond)
continue
}
c.(*net.TCPConn).SetLinger(0)
conns.Add(1)
for r := 0; (rounds <= 0 || r < rounds) && time.Now().Before(deadline); r++ {
if _, err := c.Write(out); err != nil {
errs.Add(1)
break
}
if _, err := io.ReadFull(c, in); err != nil {
errs.Add(1)
break
}
trips.Add(1)
}
c.Close()
}
}()
}
wg.Wait()
el := time.Since(start)
u1, k1 := cpuTimes()
ln.Close()
for _, c := range held {
c.Close()
}
u, k := u1-u0, k1-k0
fmt.Printf("workers=%d gomaxprocs=%d elapsed=%s payload=%d rounds=%d idle_conns=%d\n",
workers, runtime.GOMAXPROCS(0), el.Round(time.Millisecond), payload, rounds, len(held))
fmt.Printf("connections=%d (%.0f/s) round_trips=%d (%.0f/s) errors=%d\n",
conns.Load(), float64(conns.Load())/el.Seconds(), trips.Load(), float64(trips.Load())/el.Seconds(), errs.Load())
fmt.Printf("cpu_user=%.1fs cpu_kernel=%.1fs kernel_share=%.1f%%\n",
u.Seconds(), k.Seconds(), 100*k.Seconds()/(u+k).Seconds())
}
func probe(dur time.Duration) {
const tick = 100 * time.Millisecond
var worst time.Duration
var n, late int
end := time.Now().Add(dur)
for time.Now().Before(end) {
t := time.Now()
time.Sleep(tick)
over := time.Since(t) - tick
n++
if over > worst {
worst = over
}
if over > 200*time.Millisecond {
late++
}
}
fmt.Printf("probe samples=%d max_wakeup_delay=%s delays_over_200ms=%d\n", n, worst.Round(time.Millisecond), late)
}
func main() {
if len(os.Args) < 2 {
fmt.Println("usage: fserepro load|probe [flags]")
os.Exit(2)
}
fs := flag.NewFlagSet(os.Args[1], flag.ExitOnError)
workers := fs.Int("workers", runtime.NumCPU(), "concurrent client connections")
dur := fs.Duration("dur", 15*time.Second, "run time")
payload := fs.Int("payload", 4096, "bytes per request")
rounds := fs.Int("rounds", 0, "round trips per connection, 0 = one connection per worker")
idle := fs.Int("idle", 0, "extra idle connections held open during the load")
low := fs.Bool("belownormal", false, "run at below-normal priority")
fs.Parse(os.Args[2:])
if *low {
lowerPriority()
}
switch os.Args[1] {
case "load":
load(*workers, *dur, *payload, *rounds, *idle)
case "probe":
probe(*dur)
}
}
Expected Behavior
Host-only loopback TCP performance should not depend on whether a mirrored-mode WSL VM is running. These packets never leave the Windows host.
Actual Behavior
While the VM is running, the host loopback TCP path becomes much more expensive. The cost grows with the number of local ports in use. Connection-heavy workloads end up with every core spinning on a kernel spin lock.
Minimal repro results. Each run is 10–15 s with 32 workers on 32 hardware threads, load at below-normal priority:
| Scenario | WSL stopped | WSL running (mirrored) |
|---|---|---|
| 32 persistent connections, 0 idle | 271k round trips/s | 214k round trips/s |
| 32 persistent connections, 2000 idle | 251k round trips/s, probe max delay 1 ms | 137k round trips/s, probe max delay 23 ms |
| 32 persistent connections, ~8000 idle | 222k round trips/s, 3 ms | 141k round trips/s, 221 ms |
| connect + 10 round trips + close, 0 idle | 7,287 connections/s | 5,277 connections/s |
| connect + 10 round trips + close, 4000 idle | 1,015 connections/s, 19 s kernel CPU in 10 s | 118 connections/s, 224 s kernel CPU in 10 s (≈22 cores busy, 99.5% kernel) |
In the last row, kernel CPU per connection goes from about 1.9 ms to about 188 ms, roughly 100×.
Real-world impact: this was first noticed with a Go test suite that starts dozens of local proxy listeners and opens many concurrent loopback connections (about 500 established and about 5,000 in TIME_WAIT). While the WSL VM was running:
- The entire desktop froze for about 6.5 minutes. A 1-second performance-counter sampler only got scheduled every ~30 s.
- The test process used 118 s of user CPU and 3,398 s of kernel CPU.
- System totals: 85–100% privileged time, DPC + interrupt time under 3%, processor queue length up to 42.
- CPU clocks stayed normal at 5.2–5.7 GHz, with no thermal or PROCHOT throttling.
After wsl --shutdown, the same suite finished in 62 s instead of 396 s, used 27 s of kernel CPU, and the desktop stayed responsive.
The vSwitch (FSE Switch) and vEthernet (FSE HostVnic) adapters stay up after wsl --shutdown, but the contention disappears. It seems to depend on the per-port partition or reservation state that exists while the VM is running.
Hot path. Taken from a WPR CPU sampling trace of the minimal repro, symbolized with public symbols (xperf butterfly, restricted to samples in ExpWaitForSpinLock*):
afd!AfdFastConnectionSend 88.8%
tcpip!TcpTcbSend → tcpip!IppSendDatagramsCommon
tcpip!IppLoopbackEnqueue → tcpip!IppLbTransmitStackCallout → tcpip!IppLbIndicatePackets 95.7%
Fse!FseIpsClientReceivePackets → Fse!FseDemuxShimProcessInboundNblChain
Fse!FseShimsProcessSteerNblChain
Fse!FsePortPoolGetPartitionsForPort 95.7%
├─ nt!ExAcquireSpinLockShared → nt!ExpWaitForSpinLockSharedAndAcquire 9.9%
└─ Fse!FsePortPoolGetPartitionsForPortUncached 85.8%
nt!ExAcquireSpinLockExclusiveAtDpcLevel → nt!ExpWaitForSpinLockExclusiveAndAcquire 85.8%
Smaller contributors: the bind path, tcpip!InetAcquireWildcardPortUnderLock → PtLocalClientReservePortRange → Fse!FseLocalServerAcquirePorts → FsePortTrackerAcquirePorts → FsePartitionFindById (shared lock, 3%). Among non-waiting on-CPU time, the top fse.sys frame is Fse!FsePortPoolForEachReservationThatContainsPort, at about 3–5% of the process samples.
The real-world trace shows the same picture. 94% of the test process's samples are in ExpWaitForSpinLock{Exclusive,Shared}AndAcquire, entered through the same FseShimsProcessSteerNblChain → FsePortPoolGetPartitionsForPortUncached path.
Possible explanation (inferred from function names only): every loopback packet delivered by IppLbIndicatePackets is steered by FSE, which looks up the port's partitions. A cache miss falls through to ...Uncached, which takes a port-pool-wide spin lock exclusively at DISPATCH_LEVEL and appears to walk the reservations that contain the port. That walk would explain why the cost grows with the number of ports in use. Connection churn adds and removes reservations, which presumably keeps the cache cold. Because the waiters spin at DISPATCH_LEVEL, thread priority does not help, and other threads on those cores, including UI threads, are starved.
Workarounds: wsl --shutdown before running loopback-heavy workloads, switching to NAT mode, or limiting concurrency.
Diagnostic Logs
No logs attached. The standalone program above reproduces the problem in under a minute, and the call stacks above come from a WPR CPU sampling trace (wpr -start CPU) of that program. If needed, I can collect collect-wsl-logs.ps1 -LogProfile networking or a CPU trace during the repro and share it privately.
- Lingua principale
- C++
- Stelle
- 34.4k
- Fork
- 1.9k
- Merge medio
- 1g 23h
- PR unite (30g)
- 166
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di microsoft/WSL
-
feature wsl2
Difficoltà 2/5 1-3 ore Idoneità per principianti 80/100
microsoft/WSL#42306 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
feature
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
microsoft/WSL#42114 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
feature kernel wsl2
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
microsoft/WSL#41107 · 3 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
feature kernel
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
microsoft/WSL#41054 · 9 commenti · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
feature GPU
Difficoltà 1/5 1-3 ore Idoneità per principianti 70/100
microsoft/WSL#13592 · 4 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di microsoft/WSL
Issue simili
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
nextcloud/passman-android#214 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 1-3 ore Idoneità per principianti 62/100
I maintainer di solito rispondono entro 1 giorno
-
2026.11.0 fails to install on Intel macOS: requirements.txt pins cryptography==50.0.2 for Darwin x86_64Forse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Aperta
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 87/100
esphome/esphome#22892 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
sec_touch: a corrupted panel ACK ends a GET as malformed while its data frame is still comingApertabug
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
distante/esphome-components#50 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
arch-based hyprland
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
noctalia-dev/noctalia#5423 ·
I maintainer di solito rispondono entro 1 giorno