ARM64 builds never hit GHA cache despite correct scope configuration

Bắt đầu với .github/workflows/ci.yml và workflow docker/github-builder@v1.13.0, sau đó so sánh các lần chạy v0.0.5 và v0.0.6 được liên kết về hành vi cache-from/cache-to cho amd64 và arm64. Bật ghi nhật ký gỡ lỗi của BuildKit hoặc kiểm tra các điểm vào cache của workflow có thể tái sử dụng để xác định lý do scope arm64 không tìm thấy các layer đã xuất. Hoàn tất khi các bản phát hành arm64 lấy được các layer được cache theo scope dành riêng cho kiến trúc mà không ảnh hưởng đến việc cache amd64.

Description

ARM64 builds using docker/github-builder with GHA cache never retrieve cached layers, while AMD64 builds hit cache consistently (15+ CACHED layers). Both architectures use identical cache configuration and scopes, but only AMD64 benefits from caching.

Expected behaviour

Both ARM64 and AMD64 builds should retrieve cached layers from the GHA cache backend using their respective architecture-scoped entries. Expected: ARM64 build time ~40s (with cache), similar to AMD64.

Actual behaviour

ARM64 builds show 0 CACHED layers and take ~80s, while AMD64 builds show 15+ CACHED layers in ~39s, despite:

• Identical cache: true and cache-mode: max configuration
• Correct automatic scope assignment: buildkit-linux-amd64 vs buildkit-linux-arm64
• Both builds happening in same workflow run with native runners
• Cache export successfully completing on ARM64 (logs show #27 exporting to GitHub Actions Cache)

This pattern is consistent across multiple release builds.

Repository URL

https://github.com/eopo/blogger-xml-exporter

Workflow run URL

Example runs showing the issue:

• v0.0.5 release build: https://github.com/eopo/blogger-xml-exporter/actions/runs/29195568696
• v0.0.6 release build: https://github.com/eopo/blogger-xml-exporter/actions/runs/29196210866

YAML workflow

From .github/workflows/ci.yml - build job using docker/github-builder:

jobs:
  build:
    uses: docker/github-builder@v1.13.0
    with:
      output: image
      push: ${{ startsWith(github.ref, 'refs/tags/') }}
      platforms: ${{ startsWith(github.ref, 'refs/tags/') && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
      cache: true
      cache-mode: max
      meta-images: ghcr.io/${{ github.repository }}
      registry-auths: |
        {
          "ghcr.io": {
            "username": "github",
            "password": "${{ github.token }}"
          }
        }

Cache Configuration Generated by docker/github-builder:

• AMD64: --cache-from type=gha,scope=buildkit-linux-amd64 --cache-to type=gha,ignore-error=true,scope=buildkit-linux-amd64,mode=max
• ARM64: --cache-from type=gha,scope=buildkit-linux-arm64 --cache-to type=gha,ignore-error=true,scope=buildkit-linux-arm64,mode=max

Workflow logs

AMD64 Build (v0.0.5 tag release):

Total time: 39s
Cached layers: 15+

Key log lines from build output:
#1 [builder 1/7] FROM golang:1.26-alpine
#1 CACHED [builder 2/7] RUN apk add --no-cache make ...
#1 CACHED [builder 3/7] WORKDIR /build
#1 CACHED [builder 4/7] COPY Makefile .
#1 CACHED [builder 5/7] RUN make setup-css-tools
#1 CACHED [builder 6/7] COPY . .
#2 [builder 7/7] RUN make build
#3 [runtime 1/3] FROM distroless/static-debian12:nonroot
#3 CACHED [runtime 2/3] COPY --from=builder /build/bin/blogger-xml-exporter /
#3 CACHED [runtime 3/3] LABEL ...
#27 writing image ...
#27 DONE

docker buildx build command used:
--cache-from type=gha,scope=buildkit-linux-amd64 \
--cache-to type=gha,ignore-error=true,scope=buildkit-linux-amd64,mode=max

ARM64 Build (v0.0.5 tag release - same workflow run):

Total time: 80s
Cached layers: 0

Key log lines from build output:
#1 [builder 1/7] FROM golang:1.26-alpine
#1 downloading ...
#1 DONE
#2 [builder 2/7] RUN apk add --no-cache make ...
#2 DONE
#3 [builder 3/7] WORKDIR /build
#3 DONE
#4 [builder 4/7] COPY Makefile .
#4 DONE
#5 [builder 5/7] RUN make setup-css-tools
#5 DONE
[...all subsequent layers download and run, NO CACHED...]
#27 exporting to GitHub Actions Cache
#27 DONE

docker buildx build command used:
--cache-from type=gha,scope=buildkit-linux-arm64 \
--cache-to type=gha,ignore-error=true,scope=buildkit-linux-arm64,mode=max

Key Observations:

• cache-from command structure is identical for both platforms
• AMD64 successfully retrieves all cached layers from GHA
• ARM64 with buildkit-linux-arm64 scope finds ZERO cache hits
• Cache export (#27 exporting to GitHub Actions Cache) completes successfully on ARM64
• But on subsequent builds (v0.0.6): ARM64 still shows 0 hits despite cache being exported

Pattern consistency:

• v0.0.5 tag run (29195568696): AMD64 15+ CACHED, ARM64 0 CACHED (39s vs 80s)
• v0.0.6 tag run (29196210866): AMD64 15+ CACHED, ARM64 0 CACHED (22 minutes later, same pattern)

BuildKit logs

Debug logs not currently available without enabling debug mode. However, standard build output clearly shows:

• cache-from type=gha,scope=buildkit-linux-arm64 is correctly configured
• Cache retrieval attempt occurs (docker buildx sends the query)
• All layers proceed as uncached misses
• Cache export to type=gha,scope=buildkit-linux-arm64,mode=max completes without error

Additional info

Environment:

• GitHub Actions, docker/github-builder@v1.13.0
• Native ubuntu-24.04 runner for AMD64 builds
• Native ubuntu-24.04-arm runner for ARM64 builds
• Go 1.26 application with Alpine builder stage
• Multi-platform release builds (both architectures triggered on git tag events)

Root Cause Analysis:
GHA cache backend appears to have a platform-specific issue retrieving ARM64-scoped cache entries:

1. Scope configuration: ✅ Correct

   • Both platforms receive automatically-assigned architecture-specific scopes
   • buildkit-linux-amd64 vs buildkit-linux-arm64

2. Cache export: ✅ Succeeds

   • ARM64 logs show #27 exporting to GitHub Actions Cache DONE
   • Export completes without errors

3. Cache retrieval: ❌ 0% hit rate

   • AMD64: 15+ layers cached on first query
   • ARM64: 0 layers cached, despite identical query structure

Why not use shared cache scope?
Architecture-specific binaries are incompatible. Both images use CGO_ENABLED=0 to produce static Linux binaries, which are architecture-dependent. Sharing a cache scope across architectures would cause:

• Cache hits serving wrong architecture binary
• Build failures or runtime crashes
• Data corruption

Impact:

• ARM64 release builds take ~2x longer (~80s vs ~39s AMD64)
• Only affects release tag builds (not PR/main branch), so CI/CD cost is moderate
• Not a blocking issue, but significant performance regression for multi-arch releases

Reproducibility:
This should be reproducible with:

• Any Go + Alpine project using docker/github-builder@v1.13.0
• Multi-platform build configuration (linux/amd64,linux/arm64)
• GHA cache enabled (cache: true, cache-mode: max)
• Native ubuntu-24.04 and ubuntu-24.04-arm runners