Non-worker mode: thread-pool deadlock ~5s after startup under concurrent load (1.12.6, ZTS) — threads wedged in regularThread.waitForRequest / transitionToNewHandler
I maintainer di solito rispondono entro 1 giorno
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 42/100
Direzione di ricerca
Inizia da threadregular.go in waitForRequest e handleRequestWithRegularPHPThreads, quindi segui phpthread.go transitionToNewHandler e internal/state/state.go ThreadState.WaitFor. Riproduci, se possibile, il picco di traffico all'avvio e analizza i percorsi delle goroutine referenziati; il lavoro è completato quando il pool PHP non si blocca più sotto questo carico e il problema è coperto da test di regressione.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
On FrankenPHP 1.12.6 (PHP 8.4.23 ZTS, Caddy 2.11.4), in classic non-worker mode, a container intermittently deadlocks its entire PHP thread pool a few seconds after a clean startup, under the burst of concurrent traffic that a rolling deploy sends the instant the server binds. Once wedged: :8000 stops serving all requests, the Caddy admin endpoint :2019 stays fully responsive, the container never recovers (a curl :8000/up health check hangs forever), and only a restart clears it. It's probabilistic — one of two identical hosts wedges on a given deploy while the other is fine.
This is in the same area as #2250 (ThreadState.WaitFor / go_frankenphp_main_thread_is_ready), but the trigger is different: here the server binds, logs server running, and serves real traffic for ~5 s before the whole pool deadlocks — not a never-binds init hang.
Environment
- FrankenPHP v1.12.6, Caddy v2.11.4, PHP 8.4.23 (ZTS) — image
dunglas/frankenphp:1.12.6-php8.4-bookworm(amd64) - Classic / regular mode (
php_server; noworkerdirective) frankenphp { num_threads 300 max_threads 300 max_wait_time 10s }— fixed thread count. Pinningnum_threads == max_threadsto disable dynamic scaling did not prevent it.- Docker, 32-core / ~60 GB host, non-privileged, listening on
:8000; a reverse proxy cuts live traffic over to the new container immediately on deploy. - Also reproduced on 1.12.4; upgrading 1.12.4 → 1.12.6 did not fix it.
Timeline (from the container's own logs)
"FrankenPHP started 🐘" php_version=8.4.23 num_threads=300 max_threads=300 max_requests=0
"server running" name=srv0
... ~5 seconds of normal request handling (HTTP 200s) ...
<no request handled after ~T+5s; :8000 wedged; :2019 still answering; never recovers>
Goroutine profile at the deadlock (462 goroutines)
Two dominant clusters (?debug=1):
~151 PHP threads stuck in waitForRequest, reached via a cgo callback (locked to thread):
151 @ ...
# github.com/dunglas/frankenphp.(*regularThread).waitForRequest+0x346 /go/src/app/threadregular.go:110
# github.com/dunglas/frankenphp.(*regularThread).beforeScriptExecution+0x584 /go/src/app/threadregular.go:53
# github.com/dunglas/frankenphp.(*phpThread).transitionToNewHandler+0x66 /go/src/app/phpthread.go:177
# github.com/dunglas/frankenphp.(*inactiveThread).beforeScriptExecution+0x10e /go/src/app/threadinactive.go:26
# github.com/dunglas/frankenphp.(*inactiveThread).beforeScriptExecution+0xfe /go/src/app/threadinactive.go:36
# github.com/dunglas/frankenphp.go_frankenphp_before_script_execution+0x16f /go/src/app/phpthread.go:237
# _cgoexp_..._go_frankenphp_before_script_execution+0x1a /go/src/app/phpmainthread.go:280
# runtime.cgocallbackg1 / runtime.cgocallbackg / runtime.cgocallback
~148 incoming requests parked waiting for a PHP thread:
148 @ ...
# github.com/dunglas/frankenphp.handleRequestWithRegularPHPThreads+0x828 /go/src/app/threadregular.go:148
# github.com/dunglas/frankenphp.ServeHTTP+0x710 /go/src/app/frankenphp.go:427
# ...caddyhttp request pipeline...
Representative full stacks (?debug=2):
goroutine 17 [chan receive, 8 minutes, locked to thread]:
github.com/dunglas/frankenphp/internal/state.(*ThreadState).WaitFor(0x..., {0x..., 0x2, 0x2})
/go/src/app/internal/state/state.go:167 +0x158
github.com/dunglas/frankenphp.go_frankenphp_main_thread_is_ready()
/go/src/app/phpmainthread.go:238 +0x7d
goroutine 98 [select, 8 minutes, locked to thread]:
github.com/dunglas/frankenphp.(*regularThread).waitForRequest(0x...)
/go/src/app/threadregular.go:110 +0x347
github.com/dunglas/frankenphp.(*regularThread).beforeScriptExecution(0x...)
/go/src/app/threadregular.go:53 +0x585
github.com/dunglas/frankenphp.(*phpThread).transitionToNewHandler(0x...)
/go/src/app/phpthread.go:177 +0x67
github.com/dunglas/frankenphp.(*inactiveThread).beforeScriptExecution(0x...)
/go/src/app/threadinactive.go:26 +0x10f
github.com/dunglas/frankenphp.(*inactiveThread).beforeScriptExecution(0x...)
/go/src/app/threadinactive.go:36 +0xff
github.com/dunglas/frankenphp.go_frankenphp_before_script_execution(0x0?)
/go/src/app/phpthread.go:237 +0x170
Interpretation (tentative)
Looks like a thread-pool state-machine deadlock: ~half the PHP threads are wedged mid-transitionToNewHandler → waitForRequest (inside a cgo callback, locked to thread), so they never pick up the ~148 requests parked in handleRequestWithRegularPHPThreads, while go_frankenphp_main_thread_is_ready is still blocked in ThreadState.WaitFor. The trigger appears to be the burst of concurrent requests arriving in the first seconds after server running, while threads are still transitioning inactive → regular.
Notes
- Mutex profiling isn't enabled in this build (
/debug/pprof/mutexsampling period = 0), so I couldn't capture the lock holder. Happy to enableSetMutexProfileFractionand re-capture, or share the full 462-goroutine?debug=2dump (~14k lines). - Reproduction is probabilistic on deploy under live traffic. I can try to build a minimal repro (bind, then immediately hit with N concurrent requests during thread warm-up) if useful.
- Possibly related: #2250 (same
WaitFor/main_thread_is_readycode), #2205 (worker starvation after restarts).
- Lingua principale
- Go
- Stelle
- 11.4k
- Fork
- 490
- Merge medio
- 2g 21h
- PR unite (30g)
- 29
Preparare l'ambiente
- Include un Dockerfile o un file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di php/frankenphp
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
php/frankenphp#2671 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Include frankenphp version in logsForse già presa @ousamabenyounes l’ha presa 83 giorni fa. Apertaenhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
php/frankenphp#2483 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
suggestion about compile.mdForse di nuovo libera Una pull request per questa issue è stata chiusa senza essere unita. Apertaenhancement
Difficoltà 1/5 1-3 ore Idoneità per principianti 68/100
php/frankenphp#601 · 5 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 4/5 3-5 giorni Idoneità per principianti 42/100
php/frankenphp#2711 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 52/100
php/frankenphp#2710 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di php/frankenphp
Issue simili
-
unbounded value for n in `/v1/logs` can panic/OOMForse già presa Una pull request collegata a questa issue è aperta o già unita. Apertasmall item
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
canonical/pebble#1127 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 89/100
open-policy-agent/opa#9366 · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno
-
defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
nats-io/nats-server#8752 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement needs-acceptance wg/router-models-inference-runtime
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
vllm-project/semantic-router#5015 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
fileserver: precompressed sidecar's Content-Encoding and Content-Length end up on error responsesForse già presa @NitinKumar004 l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 73/100
caddyserver/caddy#8204 ·
I maintainer di solito rispondono entro 1 giorno