Containerd snapshotter does not save layer chain id info in metadata.db for locally built image for platform other than host platform using docker buildx

提供された docker buildx コマンドで問題を再現し、linux/arm64 イメージをビルドして削除した後に、metadata.db、content/blobs/sha256、overlayfs/snapshots を調査します。local-build のケースと registry-pull のケースを比較し、欠落している chain ID と保持されたレイヤーの内容が想定されたものか、バグを示しているのかを判断します。

Description

We had built an image for platform linux/arm64 using docker buildx on amd64 machine.
We were unable to see the layer chainID information for this image in metadata.db from where we could find the linkage to the layer directory number in snapshots folder for that layer.
How should we corelate the layer chain ID info with snapshot directory number then?

Also when we do docker rmi -f ImageID , it does not remove the layer contents from content/blobs/sha256 folder also not from overlayfs/snapshots folder.

Reproduce

Build an image using Dockerfile
docker buildx build --platform linux/arm64 -t my-image .
docker images --tree (fetch the layer from config of the image)
open the metadata.db from /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db and search for respective layer chainID

Expected behavior

We were expecting the image layers chainID info in metadata.db before pushing it to registry just same as we get after we pull it from registry if same image is pushed.

docker version

[root@OEL8 ~]# docker version
Client: Docker Engine - Community
 Version:           26.1.0
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        9714adc
 Built:             Mon Apr 22 17:07:54 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       c8af8eb
  Built:            Mon Apr 22 17:06:47 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
[root@OEL8 ~]#

docker info

[root@OEL8 ~]# docker info
Client: Docker Engine - Community
 Version:    26.1.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.26.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 7
 Server Version: 26.1.0
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.4.17-2136.330.7.1.el8uek.x86_64
 Operating System: Oracle Linux Server 8.9
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 3.544GiB
 Name: OEL8
 ID: 1bdb9cd9-009f-4f78-85f8-19324984ea0d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http://10.113.194.54:3128
 HTTPS Proxy: http://10.113.194.54:3128/
 No Proxy: localhost,127.0.0.1,localaddress,.localdomain.com
 Username: fanny5798
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[root@OEL8 ~]#

Additional Info

we have tried on docker 27 too still the issue exists there.