envbuilder causes OOM when pushing large image layers to cache

提供された docker run コマンド、Ollama レイヤー、および ENVBUILDER_CACHE_REPO を設定した状態で失敗を再現します。cache-key のログが現れる kaniko's pkg/executor/build.go#L441 から開始し、キャッシュの push 経路を追跡します。大きなレイヤーをキャッシュでき、envbuilder が OOM で強制終了されなければ完了です。

I have a .devcontainer with a Dockerfile:

FROM mcr.microsoft.com/vscode/devcontainers/base:ubuntu
# Install Ollama
RUN curl -fsSL https://ollama.com/install.sh | TERM=xterm sh

Ollama is about 3GB.

If I run envbuilder with ENVBUILDER_CACHE_REPO set, then the envbuilder process will cause an OOM (without CACHE_REPO, it completes):

docker run -it --rm \
    -v $PWD:/workspace \
    -e ENVBUILDER_WORKSPACE_FOLDER=/workspace \
    -e ENVBUILDER_INIT_SCRIPT=bash \
    -e ENVBUILDER_VERBOSE=1 \
    -e ENVBUILDER_CACHE_REPO="<aws-acct-id>.dkr.ecr.us-west-2.amazonaws.com/envbuilder-cache" \
    ghcr.io/coder/envbuilder
envbuilder v1.1.0+a3a6a83 - Build development environments from repositories in a container
No Dockerfile specified, looking for a devcontainer.json...
Building in Devcontainer mode using /.devcontainer/devcontainer.json
<...>
#1: RUN curl -fsSL https://ollama.com/install.sh | TERM=xterm sh
#1: Using new RunMarker command
#1: Cmd: /bin/sh
#1: Args: [-c curl -fsSL https://ollama.com/install.sh | TERM=xterm sh]
#1: Util.Lookup returned: &{Uid:0 Gid:0 Username:root Name:root HomeDir:/root}
#1: Performing slow lookup of group ids for root
#1: Running: [/bin/sh -c curl -fsSL https://ollama.com/install.sh | TERM=xterm sh]
>>> Installing ollama to /usr/local
>>> Downloading Linux arm64 bundle
######################################################################## 100.0%
<...>
>>> Install complete. Run "ollama" from the command line.
#1: Files changed [/etc <...> /var/log/lastlog]
#1: Taking snapshot of files...
#1: Adding to layer: [/ /etc <...> /var/log/lastlog]
#1: Hash components for file: /, mode: drwxr-xr-x, mtime: 2025-04-25 21:54:31.26944614 +0000 UTC, size: 1000, user-id: 0, group-id: 0
<...>
#1: Hash components for file: /var/log/lastlog, mode: -rw-rw-r--, mtime: 2025-04-25 21:55:28.376418937 +0000 UTC, size: 48568, user-id: 0, group-id: 17
#1: Deleting in layer: map[]
#1: Build: composite key for command RUN curl -fsSL https://ollama.com/install.sh | TERM=xterm sh &{[sha256:67d45b84f79ba63114ea97192913a59fff14c262ed2b83d15c357ac1b910b453 |1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin RUN curl -fsSL https://ollama.com/install.sh | TERM=xterm sh]}
#1: Build: cache key for command RUN curl -fsSL https://ollama.com/install.sh | TERM=xterm sh f4c8cf5ad4c513d32c23f27b704b5f4dcea782b9d5b3b7c96ce96867e8de03d3

The docker container exits after this. Checking dmesg shows that envbuilder was selected for oom-kill.

That last line ("Build: cache key for command") comes from kaniko, indicating something after that point is causing the excess memory use.

Possibly related issue on Kaniko github: https://github.com/GoogleContainerTools/kaniko/issues/909