macOS 14 (Apple silicon): every linux/amd64 container aborts with Rosetta "Failed to find vdso DT_HASH"

Open
#344 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
docker, linux, macos, rust

Research direction

Start with guest/kernel/build.sh and the arm64 vDSO build configuration in the kernel patches under guest/kernel/patches; the issue identifies the missing DT_HASH entry and proposes --hash-style=both for the vDSO link flags. Check the built guest kernel's vDSO dynamic section using the supplied script, then try the reported linux/amd64 container command on macOS 14. Done when Rosetta starts the container and reports x86_64; further Rosetta issues may remain.

Written by the indexing model from the issue text.

Description

Kiri version and system

0.2.1 on macOS 14.6 (23G80), Apple silicon (M3). Installed from Kiri-0.2.1-arm64.dmg. docker version shows engine 27.5.1, Kiri 0.2.1; the guest kernel is 6.18.58.

What you did
docker --context kiri run --rm --platform linux/amd64 alpine:3.19 uname -m
What happened
assertion failed [hash_table != nullptr]: Failed to find vdso DT_HASH
(Vdso.cpp:78 get_vdso_dynamic_data)

Exit code 133. alpine:latest and debian:bookworm-slim fail the same way, so no linux/amd64 container starts on this Mac. arm64 containers are fine: run, bind mounts, published ports, Compose and docker build all work.

What you expected instead

x86_64. The README lists amd64 images through Rosetta, on macOS 14+. The same command prints x86_64 under OrbStack 2.2.4 on this Mac (Rosetta turned on in its settings).

Anything else

This is not #199. That one was unhandled auxillary vector type 29 on macOS 15, and its fix (#208) is in this build. Here an older Rosetta stops earlier, on the vDSO. Rosetta for Linux on this Mac: /Library/Apple/usr/libexec/oah/RosettaLinux/rosetta, dated 2024-05-22, package com.apple.pkg.RosettaUpdateAuto 1.0.0.0.1.1721366179.

The guest kernel's arm64 vDSO has no DT_HASH. I read the vDSO's dynamic section from inside an arm64 container on both engines, on the same Mac (script below):

Engine Guest kernel DT_HASH DT_GNU_HASH amd64 container
Kiri 0.2.1 6.18.58 no yes aborts
OrbStack 2.2.4 7.2.9-orbstack yes yes runs

Likely cause. Linux dropped --hash-style=sysv from arch/arm64/kernel/vdso/Makefile: v6.6 has it on the ldflags-y line, and v6.12, v6.13, v6.15 and v6.18 don't. Since then the vDSO gets whatever the linker defaults to. guest/kernel/build.sh builds in a Debian container, where that default seems to be the GNU table only.

Possible fix, not tested (I haven't built a kernel with it): one more patch in guest/kernel/patches that links the arm64 vDSO with both tables. In v6.18 the line is:

-ldflags-y := -shared -soname=linux-vdso.so.1 \
+ldflags-y := -shared -soname=linux-vdso.so.1 --hash-style=both \

There may be more behind it on macOS 14's Rosetta once DT_HASH is back. OrbStack's kernel shows that a recent kernel can work with this Rosetta.

Workaround for now: run amd64 images through another engine's context.

Script for the table (run in an arm64 container: docker run --rm -i python:3.11-slim python3 - < vdso_check.py)
import ctypes, os, struct
libc = ctypes.CDLL(None)
libc.getauxval.restype = ctypes.c_ulong
libc.getauxval.argtypes = [ctypes.c_ulong]
base = libc.getauxval(33)  # AT_SYSINFO_EHDR: where the kernel mapped the vDSO
rd = lambda addr, n: ctypes.string_at(addr, n)
eh = rd(base, 64)
assert eh[:4] == b"\x7fELF", "no vDSO"
e_phoff = struct.unpack_from("<Q", eh, 0x20)[0]
e_phentsize, e_phnum = struct.unpack_from("<HH", eh, 0x36)
ph = [struct.unpack_from("<IIQQQQQQ", rd(base + e_phoff + i * e_phentsize, 56)) for i in range(e_phnum)]
bias = base - min(p[3] for p in ph if p[0] == 1)          # PT_LOAD p_vaddr
dyn = next(p for p in ph if p[0] == 2)                    # PT_DYNAMIC
tags, raw = set(), rd(bias + dyn[3], dyn[6])
for off in range(0, len(raw), 16):
    tag = struct.unpack_from("<q", raw, off)[0]
    if tag == 0:
        break
    tags.add(tag)
u = os.uname()
print(f"kernel={u.release} arch={u.machine} DT_HASH={'yes' if 4 in tags else 'NO'} DT_GNU_HASH={'yes' if 0x6ffffef5 in tags else 'NO'}")

Output:

kiri:     kernel=6.18.58 arch=aarch64 DT_HASH=NO DT_GNU_HASH=yes
orbstack: kernel=7.2.9-orbstack-00391-g2b298d05fbd4 arch=aarch64 DT_HASH=yes DT_GNU_HASH=yes
Dominant language
Rust
Stars
7.5k
Forks
329
Avg merge
7h 48m
Merged PRs (30d)
105

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from libracoder/kiri

All issues in libracoder/kiri

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.