[Rpi5 setup] Make Raspberry Pi cross-compiled binaries run out of the box on supported OS versions

Start with examples/raspberry_pi/setup.sh and the generated CMake toolchain file; the issue reports a manual build using a Bookworm sysroot and a glibc_compat.c shim. First resolve the maintainers’ open questions about the minimum supported OS, target devices, and sysroot source before planning the implementation. Done means the script builds binaries that run on the agreed supported Raspberry Pi OS versions without library version errors.

Summary

This proposal is based on a review of the discussion in #15014 and the related commit ([TODO: commit hash]), and follows up on the GLIBC handling that was left out there for simplicity.

1. Current state: examples/raspberry_pi/setup.sh downloads arm-gnu-toolchain-14.3.rel1 and builds against the glibc 2.40 bundled with that toolchain and GCC 14's libstdc++, both dynamically linked.
2. Problem: On OS versions that ship older libraries than the toolchain (e.g. Bookworm), the binaries fail at load time due to version mismatches not only in GLIBC_*, but also in GLIBCXX_* and CXXABI_*. In other words, this is not just a glibc issue — the C++ runtime (libstdc++) is mismatched as well.
3. Proposal: Match the target OS at build time. Link glibc against a sysroot of the lowest supported OS version, and link libstdc++ statically.
4. Goal: Binaries built by the script run on the supported targets (64-bit Raspberry Pi OS, Pi 4/5) without any extra steps, so that the GLIBC troubleshooting section in the tutorial is no longer needed.

This issue is meant to gather feedback on the approach before implementing it. The proposed approach has been verified with a manual build (see section 6).

1. Background

• An early version of PR #15014 bundled the toolchain's glibc into bundled-libs and installed it on the Pi via install_libs.sh, but this was removed to keep the script simple.
• Instead, the Troubleshooting section of the tutorial describes two workarounds:
  • Solution A (Recommended): Add the Debian sid repository on the Pi and upgrade libc6 and libstdc++6
  • Solution B (Advanced): Copy the Pi's root filesystem via rsync, use it as the sysroot, and rebuild
• Both require either changing the libraries on the Pi or having the user modify the build environment manually. This issue proposes the opposite direction: make the build match the target OS.
• Per the tutorial, the supported targets are Raspberry Pi 4/5 running 64-bit Raspberry Pi OS (Bullseye or newer). This proposal keeps that scope.

2. Reproduction

• Environment: Raspberry Pi 5, Raspberry Pi OS Bookworm (Debian 12), ldd (Debian GLIBC 2.36-9+rpt2+deb12u14) 2.36
• Build: examples/raspberry_pi/setup.sh pi5, toolchain arm-gnu-toolchain-14.3.rel1
• Runtime error:

./llama_main: /lib/aarch64-linux-gnu/libm.so.6: version `GLIBC_2.38' not found (required by ./llama_main)
./llama_main: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by ./llama_main)
./llama_main: /lib/aarch64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by ./llama_main)
./llama_main: /lib/aarch64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.15' not found (required by ./llama_main)
./llama_main: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /home/<user>/executorch-deployment/libllama_runner.so)
./llama_main: /lib/aarch64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/<user>/executorch-deployment/libllama_runner.so)
./llama_main: /lib/aarch64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.31' not found (required by /home/<user>/executorch-deployment/libllama_runner.so)
./llama_main: /lib/aarch64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.15' not found (required by /home/<user>/executorch-deployment/libllama_runner.so)

• Required versions vs. versions provided by the OS:

OS-provided versions were measured from the arm64 packages in the Debian repositories.
Bookworm: glibc 2.36, libstdc++6 12.2.0-14+deb12u1 (GCC 12) / Trixie: glibc 2.41, libstdc++6 14.2.0-19 (GCC 14)

Both binaries exceed what Bookworm provides not only for glibc but also for libstdc++ (GLIBCXX_*, CXXABI_*), so all three checks fail. The required versions are consistent with the toolchain: GLIBCXX_3.4.31/32 come from GCC 13+, and CXXABI_1.3.15 was introduced in GCC 14.

According to the table, only Trixie or newer satisfies the current binaries' requirements, which does not match the support range stated in the tutorial (Bullseye or newer). Also, while the tutorial describes this as a "GLIBC mismatch", the actual problem is that both glibc and libstdc++ are tied to the host toolchain's versions.

3. Why this is a problem

1. Users only find the fix after hitting the failure. The build succeeds and the failure only shows up when running on the Pi; the cause is hard to identify until the user finds the troubleshooting section.
2. The recommended fix (Solution A) can destabilize the system. It mixes glibc/libstdc++ from sid (unstable) into a stable release, changing system-wide libraries for the sake of a single application, which can also affect future package upgrades.
3. The safe fix (Solution B) has a high barrier to entry. It requires rsyncing the entire Pi filesystem and editing the CMake toolchain file by hand, which is why it is labeled Advanced.
4. Ultimately, users carry a burden the build should handle. If the supported OS versions are defined, matching them at build time is the natural place to solve this.

4. Proposal

The two libraries are fundamentally different, so they are handled differently.

glibc → dynamically link against a sysroot of the lowest supported version

• glibc should not be linked statically: features such as NSS, dlopen, and locales load other system shared libraries at runtime. So we keep dynamic linking and lower the version we link against.
• glibc is backward compatible through symbol versioning, so a binary linked against the lowest supported version runs on all newer versions (the same approach as Python's manylinux).
• The default is a single build for the lowest supported version, with an optional CLI flag to specify an OS release (bookworm/trixie, etc.).
• The sysroot is assembled from that release's Debian repository packages (libc6, libc6-dev, etc.).

libstdc++ / libgcc → static linking

• Both llama_main and libllama_runner.so dynamically require libstdc++. Since libstdc++ is tied to GCC, its version cannot be changed through the sysroot: the headers and template code already reference GCC 14 symbols at compile time.
• Linking with -static-libstdc++ -static-libgcc embeds it into the binaries, making them independent of the target OS's libstdc++ version.
• The static libstdc++.a shipped with the toolchain is itself built against glibc 2.40 and references some symbols that do not exist in glibc 2.36. To resolve these at link time, a small compatibility shim (glibc_compat.c) was added. [TODO: list of symbols it provides]
• However, statically linking into both llama_main and libllama_runner.so results in two copies of the C++ runtime in one process, which could potentially cause issues with exceptions or shared runtime state crossing the boundary. To avoid this, building libllama_runner.so as a static library and linking it into llama_main could be considered (question 5). Alternatively, the libstdc++ symbols embedded in the .so could be hidden with -Wl,--exclude-libs,ALL.

5. Alternatives considered

• Use a GCC version matching the target OS: The cleanest option, and it would not need the compatibility shim, but it requires maintaining a separate toolchain per OS release.
• Improve the documentation only: No implementation cost, but the burden on users remains.

The proposed approach keeps the current script structure mostly intact and adds only a sysroot as a new dependency, so I believe it does not conflict much with the simplification direction of PR #15014.

6. Verification (manual build)

I verified with a manual build that the proposed approach actually fixes the problem. Everything other than how the binaries are linked (ExecuTorch commit, toolchain, model, device, run command) was kept the same. No ExecuTorch source files were modified; only the build configuration (setup.sh and the generated CMake toolchain file) changed, plus the glibc_compat.c shim described above.

• Before: built with setup.sh pi5 as-is (toolchain-bundled glibc 2.40 sysroot) → fails with the errors in section 2 (exit=1)
• After: manually built against a Bookworm glibc 2.36 sysroot with libstdc++/libgcc statically linked → runs successfully (exit=0)

• Required versions were checked with objdump -T (dynamic symbol versions). libstdc++.so.6 no longer appears in the NEEDED entries of either binary.
• Note that the resulting binaries require up to GLIBC 2.36, so they still would not run on Bullseye (glibc 2.31). Supporting Bullseye would require a Bullseye sysroot instead.
• Binary size: llama_main 6.9M → 7.2M (due to the embedded libstdc++)
• Performance reference (Pi 5, Llama 3.2 1B): prefill 88.6 tok/s, decode 16.6 tok/s

$ cd ~/executorch-deployment
$ source setup_env.sh
$ ./llama_main --model_path ./llama3_2.pte --tokenizer_path ./tokenizer.model --seq_len 128 --prompt "What is the meaning of life?"
I tokenizers:regex.cpp:40] Registering override fallback regex
I tokenizers:regex.cpp:40] Registering override fallback regex
E tokenizers:hf_tokenizer.cpp:167] Error parsing json file: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'I'
What is the meaning of life? What is the purpose of life? What is the nature of reality? What is the meaning of existence?"

1.There is no straightforward answer to these questions, as they are complex and multifaceted. But here are a few philosophical perspectives that attempt to provide some insight:
"The meaning of life is to find your gift. The purpose of life is to find your gift. The nature of reality is to find your gift."
"A person who has never made a mistake must think he has never had one. If he has never had one, there is no mistake."
"Moby-Dick: The Bottom Line
PyTorchObserver {"prefill_token_per_sec":88.6076,"decode_token_per_sec":16.5609,"prompt_tokens":7,"generated_tokens":120,"model_load_start_ms":1793185221262,"model_load_end_ms":1793185223406,"inference_start_ms":1793185223406,"inference_end_ms":1793185230731,"model_execution_start_ms":1793185230670,"model_execution_end_ms":1793185230730,"prompt_eval_end_ms":1793185223485,"first_token_ms":1793185223485,"aggregate_sampling_time_ms":180,"aggregate_model_execution_time_ms":7142,"SCALING_FACTOR_UNITS_PER_SECOND":1000}
exit=0

Scope of verification: This was verified on Pi 5 + Bookworm. Newer releases such as Trixie are expected to work given glibc's backward compatibility; I plan to verify them, along with Pi 4 + Bookworm, in the implementation PR once the direction is agreed on.

7. Questions for maintainers

1. Does it fit the project's direction to solve this in the build script rather than in the documentation?
2. Should the lowest supported OS be Bullseye or Bookworm? (The current verification uses a Bookworm sysroot; supporting Bullseye would require a Bullseye sysroot.)
3. Will the target devices remain limited to Pi 4/5 going forward?
4. Is it acceptable to depend on an external package archive (the Debian repositories) to assemble the sysroot?
5. Would it be acceptable to build libllama_runner.so as a static library and link it into llama_main, or does it have other consumers?
6. Between a compatibility shim + static libstdc++ (this proposal) and a GCC version matching the target OS (alternative), which would you prefer from a maintenance perspective?

Once the direction is settled, I'm happy to open an implementation PR.

cc @psiddh @AdrianLundell @digantdesai @rascani