perf(butil): ARM64 (aarch64) performance and modernisation improvements across atomics, CRC32C, spin-wait hints, and fast timers

未关闭
#3,753 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 2 天内回复

@jimmystewpot 已经在做这个了。

开始于 2026年11月18日。

  • #3752 来自 @jimmystewpot —— 未关闭
  • #3754 来自 @jimmystewpot —— 未关闭
  • #3755 来自 @jimmystewpot —— 未关闭
  • #3756 来自 @jimmystewpot —— 未关闭

评估

难度
5/5
预计耗时
一周以上
新手友好度
12/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
cpp, linux, macos
领域
performance

调研方向

This is a cross-cutting ARM64 optimisation proposal spanning src/butil/atomicops_internals_arm64_gcc.h, src/butil/atomicops.h, src/butil/crc32c.cc, src/butil/processor.h, src/butil/time.h and src/butil/time.cpp. Read those files and the two already-linked pull requests (#3752 and #3756) before starting, since work is under way. Done would mean baseline -march=armv8-a builds pass, runtime dispatch works on Linux and Apple Silicon, and the stated benchmarks improve.

由索引模型根据 Issue 内容生成。

描述

Is your feature request related to a problem?

While profiling StarRocks (which relies heavily on brpc, bthread, and butil for RPC and concurrency) on modern ARM64 (aarch64) servers, CPU profiles highlighted several legacy ARM64 bottlenecks in butil where brpc was falling back to unoptimised software paths or legacy ARMv8.0 inline assembly:

  1. Legacy ldxr/stxr inline assembly and redundant dmb ish fences in src/butil/atomicops_internals_arm64_gcc.h:
    atomicops_internals_arm64_gcc.h is a 2014 Chromium import that hardcodes literal ldxr/stxr load-linked/store-conditional loops for NoBarrier_CompareAndSwap, NoBarrier_AtomicExchange, and NoBarrier_AtomicIncrement, and wraps them in full MemoryBarrier() (dmb ish) fences for Acquire_CompareAndSwap, Release_CompareAndSwap, and Barrier_AtomicIncrement (even emitting a full dmb ish when Acquire_CompareAndSwap fails). Because these are opaque assembly strings, GCC and Clang cannot emit ARMv8.1-A LSE instructions (cas, casa, casl, swp, ldadd, ldaddal) or -moutline-atomics runtime LSE helpers (__aarch64_cas*, __aarch64_ldadd*, __aarch64_swp*). Additionally, src/butil/atomicops.h routes Apple Silicon (OS_MACOSX + ARCH_CPU_ARM64) to deprecated <libkern/OSAtomic.h> routines instead of atomicops_internals_arm64_gcc.h.

  2. Missing ARM64 hardware CRC32C acceleration in src/butil/crc32c.cc:
    src/butil/crc32c.cc accelerates CRC32C on x86_64 (__SSE4_2__) and RISC-V (__riscv_zbc / __riscv_zvbc), but has no ARM64 (__aarch64__) hardware path. On ARM64, butil::crc32c::IsFastCrc32Supported() returns false and butil::crc32c::Extend() falls back to ExtendImpl<SlowCRC32Functor> (a ~1.2 GiB/s 4-byte software table lookup), despite FEAT_CRC32 and FEAT_PMULL being standard on modern ARM64 server and desktop CPUs.

  3. Ineffective yield spin-wait hint on 64-bit ARM in src/butil/processor.h:
    cpu_relax() on ARCH_CPU_ARM_FAMILY emits asm volatile("yield\n": : :"memory"). On single-threaded-per-core ARM64 CPUs (Neoverse N1/N2/V1/V2, Graviton 2/3/4, Ampere Altra, Apple Silicon), yield executes as a 1-cycle (~0.34 ns) nop, causing spin-wait loops (BT_LOOP_WHEN in bthread, mutex/butex spinloops, and work-stealing queues) to hammer the load/store unit and cache-coherency fabric at full frequency.

  4. Unserialised cntvct_el0 reads and runtime 64-bit integer divisions in src/butil/time.h / src/butil/time.cpp:
    When BUTIL_USE_CPU_FREQUENCY is enabled on __aarch64__, butil::detail::clock_cycles() reads cntvct_el0 without an isb barrier, allowing out-of-order execution to speculatively hoist counter reads ahead of earlier loads or branches (per ARM DDI 0487, Chapter D12 "The Generic Timer in AArch64 state", Section D12.2.2 "The virtual counter"). Furthermore, cpuwide_time_ns() executes two dependent 64-bit integer divisions (tsc / cpu_freq and remain * 1000000000L / cpu_freq) on every call instead of using a precomputed fixed-point multiplier and shift.

Describe the solution you'd like

Address these four areas in small, self-contained PRs that preserve full backward API/ABI compatibility and support both GCC and Clang on Linux AArch64 and Apple Silicon (__aarch64__):

  1. Modernise ARM64 butil::subtle atomics (src/butil/atomicops_internals_arm64_gcc.h, src/butil/atomicops.h):
    • Replace hardcoded ldxr/stxr inline assembly and dmb ish wrappers for Atomic32 and Atomic64 with GCC/Clang __atomic_* built-ins (__atomic_compare_exchange_n, __atomic_exchange_n, __atomic_add_fetch, __atomic_store_n, __atomic_load_n, and __atomic_thread_fence), and route COMPILER_GCC && ARCH_CPU_ARM64 ahead of OS_MACOSX.
  2. Add ARM64 hardware CRC32C (+crc and +crypto/PMULL) acceleration (src/butil/crc32c.cc):
    • Add an #if defined(__aarch64__) hardware path using <arm_acle.h> (__crc32cb, __crc32ch, __crc32cw, __crc32cd) and <arm_neon.h> (vmull_p64 3-way pipelined lane folding) with function target attributes (target("+crc") / target("+crc+crypto")) and runtime capability detection (getauxval(AT_HWCAP) on Linux, sysctlbyname on macOS) so baseline -march=armv8-a builds compile cleanly and dispatch at runtime.
  3. Use isb for cpu_relax() on 64-bit ARM (src/butil/processor.h):
    • Update cpu_relax() on ARCH_CPU_ARM64 to emit asm volatile("isb\n": : :"memory") (~15–30 cycle local pipeline pause without coherency traffic, matching MySQL, PostgreSQL, RocksDB, Folly, and OpenJDK), while keeping 32-bit ARCH_CPU_ARM_FAMILY on yield.
  4. Serialise clock_cycles() and optimise cpuwide_time_ns() (src/butil/time.h, src/butil/time.cpp):
    • Precede mrs %0, cntvct_el0 with isb in butil::detail::clock_cycles() on __aarch64__, and replace the two runtime 64-bit integer divisions in cpuwide_time_ns() with a precomputed 128-bit mul + shift scale factor initialised alongside detail::invariant_cpu_freq.

Describe alternatives you've considered

  • Requiring global -march=armv8.2-a+crc+crypto build flags: rejected so that brpc continues to build and run safely on baseline -march=armv8-a targets while using runtime detection (AT_HWCAP / -moutline-atomics) to take advantage of modern ARM64 extensions where available.

Additional context/screenshots

Verified on both Linux AArch64 (ARM Neoverse-V2, Ubuntu 24.04, GCC 13.3.0) and Apple Silicon (macOS arm64, Apple Clang 21):

  • Atomics (4-thread contention on Neoverse-V2): Barrier_AtomicIncrement improves from 46.34 ns/op to 15.39 ns/op (3.01x faster with baseline -march=armv8-a outline LSE) and 7.48 ns/op (6.20x faster with -march=armv8-a+lse); contended CAS spinlock improves from 68.01 ns/op to 29.74 ns/op (2.29x faster).
  • CRC32C (64 KiB buffers, compiled with baseline -march=armv8-a): Throughput increases from 1.20 GiB/s to 20.73 GiB/s (17.3x faster) on Neoverse-V2 and from 1.12 GiB/s to 28.20 GiB/s (25.2x faster) on Apple Silicon.
  • Spin-wait (cpu_relax() on Neoverse-V2): Replaces a 0.34 ns (1-cycle nop) yield with a 9.04 ns (~26-cycle) local isb pipeline pause.
  • Fast timer (cpuwide_time_ns() scaling math): Replaces two dependent 64-bit udiv instructions with mul/umulh + lsr, reducing scaling latency from 7.91 ns to 3.04 ns (2.6x faster) on Neoverse-V2 (18 ns to 8 ns per call on Apple Silicon).
主要语言
C++
星标
17.6k
派生
4.2k
平均合并
3 天 17 小时
30 天内合并 PR
72

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

apache/brpc 的其他 Issue

查看 apache/brpc 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。