Repeated opcache_reset() under concurrency permanently grows ZCSG(map_ptr_last), inflating every newly spawned php-fpm worker
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 45/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Ambito
- performance
Direzione di ricerca
The issue is in ext/opcache/ZendAccelerator.c and zend_persist.c, focusing on ZCSG(map_ptr_last) and CG(map_ptr_last) handling during opcache_reset(). Start by examining accel_activate() and zend_reset_cache_vars() to see how map_ptr_last is reset. The reproducer script shows how to trigger the bug with concurrent requests. Look for where cache_script_in_shared_memory() writes back to SHM. Done means verifying that repeated opcache_reset() no longer causes permanent memory growth in new workers.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Description
Description
On a busy php-fpm server, an application that calls opcache_reset() on many requests makes every newly spawned worker start bigger and bigger, without limit, until FPM is restarted.
The memory is a single large zero-filled anonymous mapping allocated by zend_map_ptr_extend() when a worker loads its first script from SHM. It is not visible to userland: on the affected production server PHP reported peak request memory of 2-38 MB while each worker's RSS was ~1.1 GB from birth.
Each opcache_reset() permanently increases ZCSG(map_ptr_last); it never shrinks. Workers born later allocate (and zero) the whole table.
Concurrency is required to reproduce. With resets happening while the pool is idle, growth is negligible (500 resets => +2 MB). With concurrent traffic and workers being spawned/reaped, the table grows on every reset.
I have not pinned down the exact code path for the concurrent case. My reading is that accel_activate() resets ZCSG(map_ptr_last) via zend_reset_cache_vars() only in the process that performs the restart, while workers that were busy keep their old (large) CG(map_ptr_last) and write it back to SHM on the next cache_script_in_shared_memory() (zend_persist.c: ZCSG(map_ptr_last) = CG(map_ptr_last);). That part is a hypothesis; the measurements below are not.
Real-world impact
A PHP application called opcache_reset() on ~4,700 requests/day (~1 every 16 s) on a shared server with ~850 req/min of traffic:
- growth measured in production: ~0.3 MB per reset (app of ~6,000 cached scripts)
- 21 hours after the last FPM restart, every newly spawned worker was ~960 MB at birth
- php-fpm total RSS: 52 GB across ~50 workers; a graceful reload brought it back to 2.4 GB
- the previous day the box (128 GB RAM) hit OOM, the kernel started killing processes and sites went down
- OPcache SHM is shared by all pools of that PHP version, so unrelated accounts on the same server were inflated too
Reproducer
Attached script: starts its own php-fpm (temp dir, own socket), serves a small app (2,500 autoloaded classes with inheritance/interfaces/traits) and sends concurrent FastCGI requests, optionally issuing opcache_reset() concurrently. Python 3.6+, stdlib only.
python3 php_opcache_reset_repro.py <php-fpm> <opcache.so|-> 600 noreset
python3 php_opcache_reset_repro.py <php-fpm> <opcache.so|-> 600 reset
It prints, before and after, the max worker RSS and the max anonymous mapping in a worker (where the map_ptr table lives).
Results (600 cycles, 6 concurrent requests each)
| PHP | Build | mode | max worker RSS | max anon mapping |
|---|---|---|---|---|
| 8.3.33 | Debian/Sury | noreset | 20 -> 22 MB | 4 -> 2 MB |
| 8.4.24 | Debian 13 | noreset | 21 -> 22 MB | 6 -> 2 MB |
| 8.5.8 | static-php-cli | noreset | 42 -> 44 MB | 19 -> 19 MB |
| 8.1.34 | cPanel ea-php | reset (598) | 19 -> 87 MB | 4 -> 57 MB |
| 8.1.34 | CloudLinux alt-php | reset (598) | 20 -> 85 MB | 4 -> 54 MB |
| 8.3.33 | Debian/Sury | reset (592) | 20 -> 94 MB | 4 -> 65 MB |
| 8.3.33 | cPanel ea-php | reset (598) | 19 -> 84 MB | 6 -> 55 MB |
| 8.4.24 | Debian 13 | reset (600) | 20 -> 97 MB | 6 -> 66 MB |
| 8.4.25 | cPanel ea-php | reset (600) | 20 -> 89 MB | 6 -> 58 MB |
| 8.4.25 | CloudLinux alt-php | reset (595) | 21 -> 87 MB | 6 -> 55 MB |
| 8.5.8 | static-php-cli | reset (597) | 43 -> 111 MB | 19 -> 61 MB |
Growth is linear and does not level off: 1,800 resets give ~200 MB (8.3.33) and ~206 MB (8.5.8) of anonymous mapping per worker.
Note this is distinct from GH-8646, whose fix (zend_map_ptr_reset() at request end when CG(interned_strings) is non-empty) only applies when OPcache is disabled. Here OPcache is enabled and the growth happens through opcache_reset().
Expected result
Repeated opcache_reset() should not permanently increase per-worker memory. After a reset the map_ptr table should return to a size proportional to the code actually cached.
Actual result
ZCSG(map_ptr_last) grows on every reset and never shrinks, so every worker spawned afterwards allocates and zero-fills an ever larger table, until php-fpm is restarted/reloaded.
PHP Version
Reproduced on 8.1.34, 8.3.33, 8.4.24, 8.4.25 and 8.5.8 (Debian, Sury, static-php-cli, cPanel ea-php, CloudLinux alt-php). The relevant code in ext/opcache/ZendAccelerator.c looks unchanged in master.
Operating System
AlmaLinux 10 (production), Debian 13 (local repro)
- Lingua principale
- C
- Stelle
- 40.4k
- Fork
- 8.2k
- Merge medio
- 2g 17h
- PR unite (30g)
- 115
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di php/php-src
-
Bug Status: Needs Triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
-
Bug Status: Needs Triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
-
Bug Status: Needs Triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Flaky hrtime.phpt test ApertaBug Category: Tests Status: Verified
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
-
Bug SAPI: fpm Status: Needs Triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
Issue simili
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
bradcypert/plum#53 ·
-
Component: GLib
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
-
Status: Opened
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
nextbsd/nextbsd-userland#285 ·