perf(butil): ARM64 (aarch64) performance and modernisation improvements across atomics, CRC32C, spin-wait hints, and fast timers

Ouverte
#3,753 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 2 jours

@jimmystewpot y travaille déjà.

Depuis le 18/11/2026.

  • #3752 par @jimmystewpot — ouverte
  • #3754 par @jimmystewpot — ouverte
  • #3755 par @jimmystewpot — ouverte
  • #3756 par @jimmystewpot — ouverte

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
12/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
cpp, linux, macos
Domaine
performance

Piste de recherche

This is a cross-cutting ARM64 optimisation proposal spanning src/butil/atomicops_internals_arm64_gcc.h, src/butil/atomicops.h, src/butil/crc32c.cc, src/butil/processor.h, src/butil/time.h and src/butil/time.cpp. Read those files and the two already-linked pull requests (#3752 and #3756) before starting, since work is under way. Done would mean baseline -march=armv8-a builds pass, runtime dispatch works on Linux and Apple Silicon, and the stated benchmarks improve.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Is your feature request related to a problem?

While profiling StarRocks (which relies heavily on brpc, bthread, and butil for RPC and concurrency) on modern ARM64 (aarch64) servers, CPU profiles highlighted several legacy ARM64 bottlenecks in butil where brpc was falling back to unoptimised software paths or legacy ARMv8.0 inline assembly:

  1. Legacy ldxr/stxr inline assembly and redundant dmb ish fences in src/butil/atomicops_internals_arm64_gcc.h:
    atomicops_internals_arm64_gcc.h is a 2014 Chromium import that hardcodes literal ldxr/stxr load-linked/store-conditional loops for NoBarrier_CompareAndSwap, NoBarrier_AtomicExchange, and NoBarrier_AtomicIncrement, and wraps them in full MemoryBarrier() (dmb ish) fences for Acquire_CompareAndSwap, Release_CompareAndSwap, and Barrier_AtomicIncrement (even emitting a full dmb ish when Acquire_CompareAndSwap fails). Because these are opaque assembly strings, GCC and Clang cannot emit ARMv8.1-A LSE instructions (cas, casa, casl, swp, ldadd, ldaddal) or -moutline-atomics runtime LSE helpers (__aarch64_cas*, __aarch64_ldadd*, __aarch64_swp*). Additionally, src/butil/atomicops.h routes Apple Silicon (OS_MACOSX + ARCH_CPU_ARM64) to deprecated <libkern/OSAtomic.h> routines instead of atomicops_internals_arm64_gcc.h.

  2. Missing ARM64 hardware CRC32C acceleration in src/butil/crc32c.cc:
    src/butil/crc32c.cc accelerates CRC32C on x86_64 (__SSE4_2__) and RISC-V (__riscv_zbc / __riscv_zvbc), but has no ARM64 (__aarch64__) hardware path. On ARM64, butil::crc32c::IsFastCrc32Supported() returns false and butil::crc32c::Extend() falls back to ExtendImpl<SlowCRC32Functor> (a ~1.2 GiB/s 4-byte software table lookup), despite FEAT_CRC32 and FEAT_PMULL being standard on modern ARM64 server and desktop CPUs.

  3. Ineffective yield spin-wait hint on 64-bit ARM in src/butil/processor.h:
    cpu_relax() on ARCH_CPU_ARM_FAMILY emits asm volatile("yield\n": : :"memory"). On single-threaded-per-core ARM64 CPUs (Neoverse N1/N2/V1/V2, Graviton 2/3/4, Ampere Altra, Apple Silicon), yield executes as a 1-cycle (~0.34 ns) nop, causing spin-wait loops (BT_LOOP_WHEN in bthread, mutex/butex spinloops, and work-stealing queues) to hammer the load/store unit and cache-coherency fabric at full frequency.

  4. Unserialised cntvct_el0 reads and runtime 64-bit integer divisions in src/butil/time.h / src/butil/time.cpp:
    When BUTIL_USE_CPU_FREQUENCY is enabled on __aarch64__, butil::detail::clock_cycles() reads cntvct_el0 without an isb barrier, allowing out-of-order execution to speculatively hoist counter reads ahead of earlier loads or branches (per ARM DDI 0487, Chapter D12 "The Generic Timer in AArch64 state", Section D12.2.2 "The virtual counter"). Furthermore, cpuwide_time_ns() executes two dependent 64-bit integer divisions (tsc / cpu_freq and remain * 1000000000L / cpu_freq) on every call instead of using a precomputed fixed-point multiplier and shift.

Describe the solution you'd like

Address these four areas in small, self-contained PRs that preserve full backward API/ABI compatibility and support both GCC and Clang on Linux AArch64 and Apple Silicon (__aarch64__):

  1. Modernise ARM64 butil::subtle atomics (src/butil/atomicops_internals_arm64_gcc.h, src/butil/atomicops.h):
    • Replace hardcoded ldxr/stxr inline assembly and dmb ish wrappers for Atomic32 and Atomic64 with GCC/Clang __atomic_* built-ins (__atomic_compare_exchange_n, __atomic_exchange_n, __atomic_add_fetch, __atomic_store_n, __atomic_load_n, and __atomic_thread_fence), and route COMPILER_GCC && ARCH_CPU_ARM64 ahead of OS_MACOSX.
  2. Add ARM64 hardware CRC32C (+crc and +crypto/PMULL) acceleration (src/butil/crc32c.cc):
    • Add an #if defined(__aarch64__) hardware path using <arm_acle.h> (__crc32cb, __crc32ch, __crc32cw, __crc32cd) and <arm_neon.h> (vmull_p64 3-way pipelined lane folding) with function target attributes (target("+crc") / target("+crc+crypto")) and runtime capability detection (getauxval(AT_HWCAP) on Linux, sysctlbyname on macOS) so baseline -march=armv8-a builds compile cleanly and dispatch at runtime.
  3. Use isb for cpu_relax() on 64-bit ARM (src/butil/processor.h):
    • Update cpu_relax() on ARCH_CPU_ARM64 to emit asm volatile("isb\n": : :"memory") (~15–30 cycle local pipeline pause without coherency traffic, matching MySQL, PostgreSQL, RocksDB, Folly, and OpenJDK), while keeping 32-bit ARCH_CPU_ARM_FAMILY on yield.
  4. Serialise clock_cycles() and optimise cpuwide_time_ns() (src/butil/time.h, src/butil/time.cpp):
    • Precede mrs %0, cntvct_el0 with isb in butil::detail::clock_cycles() on __aarch64__, and replace the two runtime 64-bit integer divisions in cpuwide_time_ns() with a precomputed 128-bit mul + shift scale factor initialised alongside detail::invariant_cpu_freq.

Describe alternatives you've considered

  • Requiring global -march=armv8.2-a+crc+crypto build flags: rejected so that brpc continues to build and run safely on baseline -march=armv8-a targets while using runtime detection (AT_HWCAP / -moutline-atomics) to take advantage of modern ARM64 extensions where available.

Additional context/screenshots

Verified on both Linux AArch64 (ARM Neoverse-V2, Ubuntu 24.04, GCC 13.3.0) and Apple Silicon (macOS arm64, Apple Clang 21):

  • Atomics (4-thread contention on Neoverse-V2): Barrier_AtomicIncrement improves from 46.34 ns/op to 15.39 ns/op (3.01x faster with baseline -march=armv8-a outline LSE) and 7.48 ns/op (6.20x faster with -march=armv8-a+lse); contended CAS spinlock improves from 68.01 ns/op to 29.74 ns/op (2.29x faster).
  • CRC32C (64 KiB buffers, compiled with baseline -march=armv8-a): Throughput increases from 1.20 GiB/s to 20.73 GiB/s (17.3x faster) on Neoverse-V2 and from 1.12 GiB/s to 28.20 GiB/s (25.2x faster) on Apple Silicon.
  • Spin-wait (cpu_relax() on Neoverse-V2): Replaces a 0.34 ns (1-cycle nop) yield with a 9.04 ns (~26-cycle) local isb pipeline pause.
  • Fast timer (cpuwide_time_ns() scaling math): Replaces two dependent 64-bit udiv instructions with mul/umulh + lsr, reducing scaling latency from 7.91 ns to 3.04 ns (2.6x faster) on Neoverse-V2 (18 ns to 8 ns per call on Apple Silicon).
Langage dominant
C++
Étoiles
17.6k
Forks
4.2k
Merge moyen
3 j 17 h
PR mergées (30 j)
72

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de apache/brpc

Toutes les issues de apache/brpc

Issues similaires

Plus d'issues C++

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.