[CommunityToolkit.Aspire.Hosting.K3s] Pod sandbox creation fails with cgroup.procs ENOENT on Windows 11, WSL 3.0.5 and Docker Desktop 4.88
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 48/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- csharp, docker, kubernetes
- Ambito
- infrastructure
Direzione di ricerca
Start with src/CommunityToolkit.Aspire.Hosting.K3s/K3sBuilderExtensions.cs, especially the container runtime arguments and K3sInitEntrypointScript described in the issue. Reproduce the failure with the listed Windows, WSL, Docker Desktop, and k3s versions, then inspect the container logs and cgroup paths. Done means pod sandboxes start successfully and CoreDNS and local-path-provisioner reach Running without cgroup errors.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Describe the bug
Using Windows 11, WSL 3.0.5 and Docker Desktop 4.88, the k3s cluster created by CommunityToolkit.Aspire.Hosting.K3s no longer runs reliably.
The Kubernetes node reports Ready, but pod sandbox creation repeatedly fails. CoreDNS, local-path-provisioner, the SPIRE CSI driver and other workloads remain in ContainerCreating, Pending or init states.
A representative failure is:
Failed to create sandbox for pod
failed to create containerd task
failed to create shim task
OCI runtime create failed
runc create failed
unable to apply cgroup configuration
failed to write 49865:
write /sys/fs/cgroup/kubepods/burstable/podbd669241-1f1f-4bf9-a4d9-7d7aea8c4329/5f73d605c09ef05dc6a7ca044fc2a22fa087a5f864dc14301ddc8f0e5b8ac857/cgroup.procs:
no such file or directory
The same failure occurs for besteffort pods, including local-path-provisioner and the SPIRE CSI driver.
kubelet also cannot move the k3s process into its own cgroup:
Failed to ensure state containerName="/k3s"
failed to move PID 7
(in "/docker/04a15f167a62e22903e6733541aec46384c8d1663402c4d19e29864e7e2d66fb")
to "/k3s":
failed to write 7:
write /sys/fs/cgroup/k3s/cgroup.procs:
no such file or directory
However, /sys/fs/cgroup/k3s/cgroup.procs exists and /sys/fs/cgroup/k3s/cgroup.type reports domain.
The application does not explicitly configure the cgroup namespace or mount /sys/fs/cgroup.
These settings are added by:
https://github.com/CommunityToolkit/Aspire/blob/main/src/CommunityToolkit.Aspire.Hosting.K3s/K3sBuilderExtensions.cs
.WithContainerRuntimeArgs("--privileged")
.WithContainerRuntimeArgs("--init")
.WithContainerRuntimeArgs("--userns=host")
.WithContainerRuntimeArgs("--cgroupns=host")
.WithContainerRuntimeArgs("--volume=/sys/fs/cgroup:/sys/fs/cgroup:rw")
.WithContainerRuntimeArgs("--tmpfs=/run", "--tmpfs=/var/run")
The same file defines K3sInitEntrypointScript, which moves processes from the visible root cgroup into /sys/fs/cgroup/init and enables the available cgroup v2 controllers.
This is a hypothesis rather than a confirmed root cause, but the problem may involve combining:
--cgroupns=host
--volume=/sys/fs/cgroup:/sys/fs/cgroup:rw
with the current WSL and Docker Desktop cgroup v2 implementation. Inside the k3s container, the k3s process is reported under /docker/<container-id>, while the explicitly mounted cgroup filesystem appears to expose a different WSL/Docker Desktop hierarchy. This could produce an inconsistent cgroup namespace and filesystem view, causing writes to an existing cgroup.procs file to return ENOENT.
The entrypoint script also suppresses cgroup setup failures using 2>/dev/null || true, potentially allowing k3s to start and report the node as Ready despite unsuccessful or incomplete cgroup initialisation.
Environment:
Windows edition: Windows Enterprise
Windows version: 25H2
Windows build: 26200.9711
WSL version: 3.0.5.0
WSL kernel: 6.18.58.1-1
WSLg version: 1.0.79
Ubuntu: 24.04.5 LTS
systemd: enabled
Docker Desktop: 4.88.0
Docker Engine: 29.7.2
Docker containerd: 2.3.3
Docker runc: 1.4.3
Docker cgroup version: 2
Docker cgroup driver: cgroupfs
k3s image: rancher/k3s:v1.36.0-k3s1
Kubernetes: v1.36.0+k3s1
k3s containerd: 2.2.3-k3s1
Regression
Possible. The setup worked before the managed WSL and Docker Desktop upgrade. The last confirmed working CommunityToolkit.Aspire.Hosting.K3s version was also 13.5.0; the toolkit package was not changed when the problem appeared. The environmental change was the WSL and Docker Desktop upgrade.
Steps to reproduce
1. On Windows 11 Enterprise 25H2, install or upgrade to WSL 3.0.5.0 with kernel 6.18.58.1 and Docker Desktop 4.88.0.
2. Use Ubuntu 24.04.5 LTS under WSL with systemd enabled.
3. Create a .NET 10 Aspire AppHost referencing `CommunityToolkit.Aspire.Hosting.K3s` version `13.5.0`.
4. Add a k3s cluster:
var k3sCluster = builder
.AddK3sCluster("kubernetes", apiServerPort: 16443)
.WithDisabledComponent("traefik");
5. Start the Aspire AppHost.
6. Identify the generated k3s container:
docker ps --format 'Name={{.Names}} Image={{.Image}} Status={{.Status}}'
7. Confirm the Kubernetes node reports `Ready`:
docker exec <k3s-container> kubectl \
--kubeconfig=/etc/rancher/k3s/k3s.yaml \
get nodes -o wide
8. Inspect the pods:
docker exec <k3s-container> kubectl \
--kubeconfig=/etc/rancher/k3s/k3s.yaml \
get pods -A -o wide
9. Observe CoreDNS and local-path-provisioner remaining in `ContainerCreating`.
10. Inspect the k3s container logs:
docker logs --tail 500 <k3s-container>
11. Observe repeated `cgroup.procs: no such file or directory` failures from runc and kubelet.
Expected behavior
The k3s cluster should create pod sandboxes successfully on the supported Windows 11, WSL and Docker Desktop environment.
The Kubernetes node should report Ready, and core components such as CoreDNS and local-path-provisioner should reach Running without cgroup-related runc failures.
Screenshots
No response
IDE and version
VS Code
IDE version
No response
Nuget packages
<Project Sdk="Aspire.AppHost.Sdk/13.6.5">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<IsAspireHost>true</IsAspireHost>
<AspireUseCliBundle>true</AspireUseCliBundle>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CommunityToolkit.Aspire.Hosting.K3s" Version="13.5.0" />
</ItemGroup>
</Project>
Additional context
aspire-k3s-cgroup-diagnostics.txt
Help us help you
Possible - can work with the author(s) of this extension 😎
- Lingua principale
- C#
- Stelle
- 634
- Fork
- 206
- Merge medio
- 1g 10h
- PR unite (30g)
- 104
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di CommunityToolkit/Aspire
-
[14.0] Remove Java ResourceForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
CommunityToolkit/Aspire#1622 · 6 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
[14.0] Remove Rust resourceForse già presa @aaronpowell l’ha presa 1 giorno fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
CommunityToolkit/Aspire#1620 · 6 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Perl - Feature Parity On Certificate BehaviorForse già presa @Omnideth l’ha presa oggi. Apertafeature request :mailbox_with_mail:
Difficoltà 4/5 3-5 giorni Idoneità per principianti 54/100
CommunityToolkit/Aspire#2538 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
CommunityToolkit/Aspire#2533 ·
I maintainer di solito rispondono entro 1 giorno
-
Sql ProjectsForse già presa @ErikEJ l’ha presa 7 giorni fa. Aperta
CommunityToolkit/Aspire#2435 · 1 commento · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di CommunityToolkit/Aspire
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
conductor-oss/csharp-sdk#183 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno
-
CreateNewLangProj throws when the default analysis writing system's ICU locale differs from its tagAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
-
kind/bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
unoplatform/uno#26555 ·
I maintainer di solito rispondono entro 1 giorno
-
Search button on a book does not retry without the author when the full query finds nothingForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
Listenarrs/Listenarr#1173 ·
I maintainer di solito rispondono entro 1 giorno