[CommunityToolkit.Aspire.Hosting.K3s] Pod sandbox creation fails with cgroup.procs ENOENT on Windows 11, WSL 3.0.5 and Docker Desktop 4.88

Abierto
#2,436 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
csharp, docker, kubernetes

Línea de trabajo

Start with src/CommunityToolkit.Aspire.Hosting.K3s/K3sBuilderExtensions.cs, especially the container runtime arguments and K3sInitEntrypointScript described in the issue. Reproduce the failure with the listed Windows, WSL, Docker Desktop, and k3s versions, then inspect the container logs and cgroup paths. Done means pod sandboxes start successfully and CoreDNS and local-path-provisioner reach Running without cgroup errors.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Describe the bug

Using Windows 11, WSL 3.0.5 and Docker Desktop 4.88, the k3s cluster created by CommunityToolkit.Aspire.Hosting.K3s no longer runs reliably.

The Kubernetes node reports Ready, but pod sandbox creation repeatedly fails. CoreDNS, local-path-provisioner, the SPIRE CSI driver and other workloads remain in ContainerCreating, Pending or init states.

A representative failure is:

Failed to create sandbox for pod
failed to create containerd task
failed to create shim task
OCI runtime create failed
runc create failed
unable to apply cgroup configuration
failed to write 49865:
write /sys/fs/cgroup/kubepods/burstable/podbd669241-1f1f-4bf9-a4d9-7d7aea8c4329/5f73d605c09ef05dc6a7ca044fc2a22fa087a5f864dc14301ddc8f0e5b8ac857/cgroup.procs:
no such file or directory

The same failure occurs for besteffort pods, including local-path-provisioner and the SPIRE CSI driver.

kubelet also cannot move the k3s process into its own cgroup:

Failed to ensure state containerName="/k3s"
failed to move PID 7
(in "/docker/04a15f167a62e22903e6733541aec46384c8d1663402c4d19e29864e7e2d66fb")
to "/k3s":
failed to write 7:
write /sys/fs/cgroup/k3s/cgroup.procs:
no such file or directory

However, /sys/fs/cgroup/k3s/cgroup.procs exists and /sys/fs/cgroup/k3s/cgroup.type reports domain.

The application does not explicitly configure the cgroup namespace or mount /sys/fs/cgroup.

These settings are added by:


https://github.com/CommunityToolkit/Aspire/blob/main/src/CommunityToolkit.Aspire.Hosting.K3s/K3sBuilderExtensions.cs

.WithContainerRuntimeArgs("--privileged")
.WithContainerRuntimeArgs("--init")
.WithContainerRuntimeArgs("--userns=host")
.WithContainerRuntimeArgs("--cgroupns=host")
.WithContainerRuntimeArgs("--volume=/sys/fs/cgroup:/sys/fs/cgroup:rw")
.WithContainerRuntimeArgs("--tmpfs=/run", "--tmpfs=/var/run")

The same file defines K3sInitEntrypointScript, which moves processes from the visible root cgroup into /sys/fs/cgroup/init and enables the available cgroup v2 controllers.

This is a hypothesis rather than a confirmed root cause, but the problem may involve combining:

--cgroupns=host
--volume=/sys/fs/cgroup:/sys/fs/cgroup:rw

with the current WSL and Docker Desktop cgroup v2 implementation. Inside the k3s container, the k3s process is reported under /docker/<container-id>, while the explicitly mounted cgroup filesystem appears to expose a different WSL/Docker Desktop hierarchy. This could produce an inconsistent cgroup namespace and filesystem view, causing writes to an existing cgroup.procs file to return ENOENT.

The entrypoint script also suppresses cgroup setup failures using 2>/dev/null || true, potentially allowing k3s to start and report the node as Ready despite unsuccessful or incomplete cgroup initialisation.

Environment:

Windows edition: Windows Enterprise
Windows version: 25H2
Windows build: 26200.9711

WSL version: 3.0.5.0
WSL kernel: 6.18.58.1-1
WSLg version: 1.0.79

Ubuntu: 24.04.5 LTS
systemd: enabled

Docker Desktop: 4.88.0
Docker Engine: 29.7.2
Docker containerd: 2.3.3
Docker runc: 1.4.3
Docker cgroup version: 2
Docker cgroup driver: cgroupfs

k3s image: rancher/k3s:v1.36.0-k3s1
Kubernetes: v1.36.0+k3s1
k3s containerd: 2.2.3-k3s1
Regression

Possible. The setup worked before the managed WSL and Docker Desktop upgrade. The last confirmed working CommunityToolkit.Aspire.Hosting.K3s version was also 13.5.0; the toolkit package was not changed when the problem appeared. The environmental change was the WSL and Docker Desktop upgrade.

Steps to reproduce
1. On Windows 11 Enterprise 25H2, install or upgrade to WSL 3.0.5.0 with kernel 6.18.58.1 and Docker Desktop 4.88.0.
2. Use Ubuntu 24.04.5 LTS under WSL with systemd enabled.
3. Create a .NET 10 Aspire AppHost referencing `CommunityToolkit.Aspire.Hosting.K3s` version `13.5.0`.
4. Add a k3s cluster:


var k3sCluster = builder
    .AddK3sCluster("kubernetes", apiServerPort: 16443)
    .WithDisabledComponent("traefik");


5. Start the Aspire AppHost.
6. Identify the generated k3s container:


docker ps --format 'Name={{.Names}} Image={{.Image}} Status={{.Status}}'


7. Confirm the Kubernetes node reports `Ready`:


docker exec <k3s-container> kubectl \
  --kubeconfig=/etc/rancher/k3s/k3s.yaml \
  get nodes -o wide


8. Inspect the pods:


docker exec <k3s-container> kubectl \
  --kubeconfig=/etc/rancher/k3s/k3s.yaml \
  get pods -A -o wide


9. Observe CoreDNS and local-path-provisioner remaining in `ContainerCreating`.
10. Inspect the k3s container logs:


docker logs --tail 500 <k3s-container>


11. Observe repeated `cgroup.procs: no such file or directory` failures from runc and kubelet.
Expected behavior

The k3s cluster should create pod sandboxes successfully on the supported Windows 11, WSL and Docker Desktop environment.

The Kubernetes node should report Ready, and core components such as CoreDNS and local-path-provisioner should reach Running without cgroup-related runc failures.

Screenshots

No response

IDE and version

VS Code

IDE version

No response

Nuget packages
<Project Sdk="Aspire.AppHost.Sdk/13.6.5">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <IsAspireHost>true</IsAspireHost>
    <AspireUseCliBundle>true</AspireUseCliBundle>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="CommunityToolkit.Aspire.Hosting.K3s" Version="13.5.0" />
  </ItemGroup>
</Project>
Additional context

aspire-k3s-cgroup-diagnostics.txt

Help us help you

Possible - can work with the author(s) of this extension 😎

Lenguaje dominante
C#
Estrellas
634
Forks
206
Merge medio
1 d 10 h
PR fusionados (30 d)
104

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de CommunityToolkit/Aspire

Todos los issues de CommunityToolkit/Aspire

Issues similares

Más issues de C#

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.