sysbox-fs 0.7.1 sometimes hangs at startup: ShmSysctlUserNamespaced forks without exec
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 64/100
Direzione di ricerca
Start in sysbox-fs: locate handlerService.Setup and the linuxUtils.ShmSysctlUserNamespaced() call that runs right after the FUSE dir log line, then read CreateUsernsProcess to see the raw SYS_CLONE fork without exec. Compare with how nsenter re-execs via /proc/self/exe, and with spawning through os/exec + SysProcAttr Cloneflags and ID mappings. Done = the shm check runs in an exec'd child (or the syscall.Wait4 wait is bounded), the "Host supports writes to /proc/sys/kernel/shm*" line appears on startup, and existing tests pass.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
On sysbox-ce 0.7.1, sysbox-fs sometimes hangs during startup and systemd kills it after TimeoutStartSec=10 ("start operation timed out"). sysbox.service then fails with result dependency and /run/sysbox/sysfs.sock never appears. It happens in roughly 1 of 30 fresh installs on GitHub-hosted runners; restarting sysbox-fs recovers it.
Environment
- GitHub Actions
ubuntu-24.04runners, kernel6.17.0-1022-azure, systemd 255 - Docker 29.8.2 (containerd image store), configured with the
sysbox-runcruntime,bipanddefault-address-poolsbefore Sysbox is installed, so the installer doesn't restart Docker sysbox-ce_0.7.1.linux_amd64.deb(sysbox-fs commitc3d2ebc65102e32e74e383675f03b45556326888)
What the journal shows
A failing start stops right after FUSE dir, with no error:
sysbox-fs[3388]: level=info msg="FUSE dir = /var/lib/sysboxfs"
systemd[1]: sysbox-fs.service: start operation timed out. Terminating.
systemd[1]: sysbox-fs.service: Failed with result 'timeout'.
systemd[1]: Dependency failed for sysbox.service - Sysbox container runtime.
A good start goes on within the same second, and the restarted instance on another runner did exactly that:
sysbox-fs[3457]: level=info msg="FUSE dir = /var/lib/sysboxfs"
sysbox-fs[3457]: level=info msg="Host supports writes to /proc/sys/kernel/shm* from user-ns"
sysbox-fs[3457]: level=info msg="IOvec memParser elected"
sysbox-fs[3457]: level=info msg="Listening on /run/sysbox/sysfs.sock"
sysbox-fs[3457]: level=info msg="Ready ..."
Likely cause
The failing start never logs the result of linuxUtils.ShmSysctlUserNamespaced(), which handlerService.Setup calls right after FUSE dir. That check calls CreateUsernsProcess, which forks with a raw SYS_CLONE and no exec, and the child then runs Go code (os.ReadFile, time.Sleep, logrus, os.OpenFile). After a fork in a multi-threaded Go process only the forking thread exists in the child, so if another thread held a runtime or allocator lock at that moment, the child can block forever. The parent waits in syscall.Wait4(pid, ...) with no timeout, so sysbox-fs hangs until systemd kills it.
Running the check in a re-exec'd child (as nsenter already does, through /proc/self/exe), or through os/exec with SysProcAttr Cloneflags and ID mappings, would avoid the fork-without-exec; bounding the wait would at least turn a hang into an error.
Workaround
A drop-in that restarts it on failure, installed before the package:
sudo mkdir -p /etc/systemd/system/sysbox-fs.service.d
printf '[Service]\nRestart=on-failure\nRestartSec=1\n' | sudo tee /etc/systemd/system/sysbox-fs.service.d/restart.conf
With it, 32 of 32 parallel installs came up; on one of them sysbox-fs hit the timeout and its restart started normally.
- Lingua principale
- Shell
- Stelle
- 3.9k
- Fork
- 237
- Merge medio
- 2g 9h
- PR unite (30g)
- 1
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di nestybox/sysbox
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 52/100
-
sysbox-fs zombie reaper can steal fusermount3 children during concurrent container creationForse già presa @rogerhu l’ha presa 29 giorni fa. Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
-
0.7.1: /sys/class/net inside a sysbox container lists the host's interfaces, not the container'sAperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 48/100
Tutte le issue di nestybox/sysbox
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
ophub/amlogic-s9xxx-armbian#3752 ·
I maintainer di solito rispondono entro 1 giorno
-
prio:low type:chore
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 82/100
davidsneighbour/dotfiles#602 ·
I maintainer di solito rispondono entro 3 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 79/100
I maintainer di solito rispondono entro 1 giorno
-
[cmocka] update to 2.0.2Apertaupdate-request
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
msys2/MINGW-packages#33004 ·
I maintainer di solito rispondono entro 1 giorno
-
bug small tooling
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno