CI: add a big-endian job (s390x, cross-compiled, tests under qemu-user)
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 45/100
Direzione di ricerca
Start with cmake/ci.cmake and .github/workflows/ubuntu.yml to see how existing CI targets are defined, then check the s390x cross-compile and qemu-user commands in the issue's sketch, which are not yet run as written. Decide how much of the suite to run, since the runtime figures are in the issue. Done means a big-endian job builds all test binaries and passes under qemu-user, and on a branch that reverts #6074 the build fails in tests/src/unit-alt-string.cpp.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Description
No CI job builds the library or its tests for a big-endian target. Every job in .github/workflows/ (Ubuntu x86-64, the native ci_test_arm64, macOS, Windows, Emscripten, CUDA, ...) runs on a little-endian target. The library has big-endian code paths: the byte swaps of the binary formats (CBOR, MessagePack, UBJSON, BJData, BSON, BON8), the SWAR scans guarded by JSON_LITTLE_ENDIAN_TARGET, and json_view, whose save()/load() do not exist on big-endian targets by design (basic_json_document::load: "Little-endian targets only."). None of this is compiled in CI on such a target.
That gap let a test break the build on big-endian targets without anyone noticing: tests/src/unit-alt-string.cpp called alt_editable_document::load(d.save()) without the little-endian guard that the other image tests have, so the test suite of develop did not compile on s390x. A bug hunt found it by hand; #6074 fixed it. Packagers who build and run the tests on s390x (Debian, Fedora, Ubuntu and SUSE all ship s390x) would have hit it with the 3.13.0 release.
Since when: always (no big-endian job has existed). The concrete breakage came with the json_view alt-string test and was fixed by #6074 (merged 2026-11-11).
Security: Not a vulnerability: a CI coverage gap.
Related: #6074 (the breakage, which says a CI job "is not part of this PR"), #5920 (no save()/load() on big-endian targets).
Possible fix: a job that cross-compiles the tests natively for s390x and runs them under qemu-user. Cross-compiling keeps the compile at native speed; only the test binaries run emulated. Building inside an emulated s390x container (uraimo/run-on-arch-action, or docker/setup-qemu-action plus an s390x/ubuntu image) would also run the compiler under QEMU, which is much slower for a suite of 180 test binaries; I did not measure that variant.
s390x rather than big-endian ppc64: Debian and Ubuntu ship the s390x cross compiler (g++-s390x-linux-gnu) and qemu-user in their main archives; big-endian ppc64 is only a Debian ports architecture (ppc64el, which they ship, is little-endian).
A sketch in the style of ubuntu.yml, with a target in cmake/ci.cmake like the other jobs:
add_custom_target(ci_test_big_endian
COMMAND ${CMAKE_COMMAND}
-DCMAKE_BUILD_TYPE=Debug -GNinja
-DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=s390x
-DCMAKE_CXX_COMPILER=s390x-linux-gnu-g++
"-DCMAKE_CROSSCOMPILING_EMULATOR=qemu-s390x\;-L\;/usr/s390x-linux-gnu"
-DJSON_BuildTests=ON -DJSON_FastTests=ON
-S${PROJECT_SOURCE_DIR} -B${PROJECT_BINARY_DIR}/build_big_endian
COMMAND ${CMAKE_COMMAND} --build ${PROJECT_BINARY_DIR}/build_big_endian
COMMAND cd ${PROJECT_BINARY_DIR}/build_big_endian && ${CMAKE_CTEST_COMMAND} --parallel ${N} --output-on-failure -LE git_required
COMMENT "Cross-compile for s390x (big-endian) and run the tests under qemu-user"
)
# Big-endian target: the tests are cross-compiled for s390x and run under
# qemu-user (only the test binaries are emulated, not the compiler).
ci_test_big_endian:
runs-on: ubuntu-26.04
container: debian:trixie
steps:
- name: Install the s390x cross compiler and qemu-user
run: |
apt-get update
apt-get install -y --no-install-recommends g++-s390x-linux-gnu qemu-user git ca-certificates
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Get latest CMake and ninja
uses: lukka/get-cmake@f86dd60acae0d132056913e248bdabf0e0580b7a # v4.4.5
- name: Check that the target is big-endian
run: echo | s390x-linux-gnu-g++ -dM -E - | grep -q '__BYTE_ORDER__ __ORDER_BIG_ENDIAN__'
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_big_endian
(Not run as written: I ran the same cmake configure, build and ctest commands directly in the container, not through a ci.cmake target or a workflow, so the \; escaping of the emulator list in add_custom_target needs a check. The -LE git_required follows the note for exported trees; inside the CI checkout it could be dropped. git is needed to download the test data.)
Runtime (measured; Debian trixie arm64 container on an Apple M1 Max under colima, limited to 4 CPUs like a GitHub runner, GCC 14.2 cross compiler, Debug, JSON_FastTests=ON, test data from a local copy):
| step | time |
|---|---|
apt-get install of the toolchain and qemu-user |
39 s |
build of all 181 test binaries, -j 4 |
592 s |
ctest -j 4 -LE git_required, 197 tests |
653 s (all passed) |
test-unicode_cpp11 alone takes 404 s under qemu, so it sets the wall time of the test step; the next slowest are test-json_patch_cpp11 (62 s) and test-ordered_json_cpp11 (56 s). The binary-format and json_view tests (CBOR, MessagePack, UBJSON, BJData, BSON, BON8, all json_view* tests and the binary writer tests: 40 tests) take 410 s of CPU time together. A GitHub-hosted x86-64 runner compiles slower than an M1 Max core and also emulates s390x, so expect the job to take more than the 21 minutes above, roughly 30 to 45 minutes (an estimate, not measured on GitHub). Options to shorten it:
- Compile all, run a subset: build everything (this is what would have caught #6074), then
ctest -R 'cbor|msgpack|ubjson|bjdata|bson|bon8|binary|json_view|alt-string|serialization|class_parser'. That keeps the endianness-relevant tests and dropstest-unicode(the code it covers is byte-order independent). - Exclude only the slow one:
ctest -E 'test-unicode', which cuts the test step to about the time of the next slowest tests divided over 4 jobs. - Run the job only on pushes to develop and on a schedule, not on every PR push. I would still run at least the build on PRs, since compiling is what broke.
With json_view images not available on big-endian targets, test-json_view_image_cpp11 there checks only that save()/load() do not exist (0.04 s in the run above); that is intended.
Test: the job itself. To check that it catches the #6074 kind of breakage, run it once on a branch that reverts #6074; the build must fail in unit-alt-string.cpp (see the error below).
Reproduction steps
How the breakage happened and how it shows on a big-endian target:
- Check out the commit before #6074 (
e0b76bac31^). - In a Debian trixie container:
apt-get install g++-s390x-linux-gnu. s390x-linux-gnu-g++ -std=c++11 -fsyntax-only -w -I include -I tests/thirdparty/doctest -I tests/src tests/src/unit-alt-string.cpp- Every CI job passed on that commit, because none compiles for a big-endian target.
Expected vs. actual results
- Expected: a CI job compiles the test suite for a big-endian target, so a test that uses a little-endian-only API without the guard fails CI.
- Actual: CI was green; the test suite did not compile on s390x until #6074, found only by a manual cross build.
Minimal code example
// tests/src/unit-alt-string.cpp before #6074 (line 534): no little-endian guard
const alt_editable_document loaded = alt_editable_document::load(d.save());
Error messages
Before #6074 (e0b76bac31^), s390x cross compiler (shortened aka types):
tests/src/unit-alt-string.cpp:534:69: error: 'load' is not a member of 'alt_editable_document' {aka 'nlohmann::json_abi_v3_12_0::basic_json_document<nlohmann::json_abi_v3_12_0::basic_json<std::map, std::vector, alt_string, ...>, nlohmann::json_abi_v3_12_0::view_mode::editable>'}
tests/src/unit-alt-string.cpp:534:76: error: 'using alt_editable_document = class nlohmann::json_abi_v3_12_0::basic_json_document<...>' {aka 'class nlohmann::json_abi_v3_12_0::basic_json_document<...>'} has no member named 'save'
develop (5bad6c518b), the proposed configuration:
build (-j 4): 592 s, 181 test binaries
100% tests passed, 0 tests failed out of 197
Total Test time (real) = 652.71 sec
Compiler and operating system
s390x-linux-gnu-g++ (Debian 14.2.0-19) 14.2.0 with qemu-user, Debian trixie arm64 container (colima) on macOS 27.0.1, Apple M1 Max
Library version
develop (5bad6c518b); 3.12.0: n/a (CI gap; the specific breakage did not exist in 3.12.0)
Found in a bug hunt on develop (2026-11-10); drafted with Claude Code.
- Lingua principale
- C++
- Stelle
- 50.9k
- Fork
- 7.5k
- Merge medio
- 1g 3h
- PR unite (30g)
- 217
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di nlohmann/json
-
documentation kind: bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
kind: bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
kind: bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
kind: bug performance
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
dump(): the recursive frame grew from 480 to 2304 bytes in unoptimized builds, so 128 levels need 288 KiB of stackForse già presa @nlohmann l’ha presa oggi. Apertakind: enhancement/improvement
nlohmann/json#6116 · 1 assegnatario ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di nlohmann/json
Issue simili
-
Favicons are coolApertaenhancement frontend
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
-
Spot Capacity Round Trips tests accept the two largest rising runsForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
intel/torch-xpu-ops#6197 ·
I maintainer di solito rispondono entro 1 giorno
-
project: hipblaslt
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
ROCm/rocm-libraries#15746 ·
I maintainer di solito rispondono entro 2 giorni
-
Needs: Triage :mag:
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
react/react-native#60000 ·
I maintainer di solito rispondono entro 2 giorni