Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Ship the aarch64-verified NEON kernels to mobile: Apple targets + Android JNI for skainet-backend-native-cpu (measured 21 → 1.0 → 0.11 tok/s cliff)

Đang mở
  • Có gì không ổn?
#920 4 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
32/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Ít trao đổi
Công nghệ
android, c, cmake, ios, kotlin
Lĩnh vực
backend, mobile-dev, performance

Hướng nghiên cứu

Start with native/src/*.c, CMakeLists.txt, skainet_kernels.def, and the native kernel provider and parity tests in nativeMain/nativeTest; then read the existing cross-Arm build pattern and publish.yml. Decide which platform track to tackle first, since Apple packaging, registration, and CI differ from the Android JNI route. Done means mobile builds can link and register the optimized kernels, Q4_0 has a NEON path, and parity/performance can be exercised on the target.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

The numbers

Same model (SmolLM2-135M-Instruct Q8_0), same QuantPolicy.NATIVE_OPTIMIZED, same
commonMain inference code, SKaiNET 0.38.0:

Platform Decode speed One 44-token generation
JVM desktop (Apple Silicon, JDK 21, Vector API) 21 tok/s ~2 s
Android, physical phone 1.0 tok/s ~44 s
iOS simulator (Apple Silicon) 0.11 tok/s ~6 min

Output is correct everywhere — this is purely a kernel-availability gap, and it is
what keeps our shipped app's AI feature switched off on phones (we gate on
3 tok/s at startup; both mobile platforms fail the gate). Roughly 200x between JVM
and Kotlin/Native on comparable CPUs.

Why it happens

skainet-backend-native-cpu publishes only -jvm, -linuxx64, -linuxarm64.
There is no Apple or Android artifact, Android has no java.lang.foreign / Vector
API, and every non-JVM PlatformCpuOpsFactory actual registers only
ScalarKernelProvider (priority 0). So both mobile platforms run scalar Kotlin
matmul while the JVM gets SIMD.

Why this is closer than it looks

Most of the hard part already exists in-tree:

  • The C kernels (native/src/*.c) are portable C11 with NEON bodies gated on
    __ARM_NEON / __ARM_FEATURE_DOTPROD — fp32, q8_0, q4k, q5k, q6k — and carry an
    "AARCH64-VERIFIED" banner: parity-checked under qemu-aarch64 and on a physical
    Cortex-A55, -march=armv8.2-a+fp16+dotprod, deliberately no +i8mm.
  • CMakeLists.txt keys the -march flags off CMAKE_SYSTEM_PROCESSOR, which NDK
    and Apple toolchains both set correctly — the flags come for free.
  • The Kotlin/Native side (NativeKnKernelProvider + wrappers + the nativeTest
    parity tests) lives in target-agnostic nativeMain/nativeTest and should
    compile for Apple/Android-native targets unchanged.
  • The publish workflow already runs on a macOS runner.

What actually needs doing

We'd suggest two tracks, iOS first (it reuses the verified K/N cinterop path
as-is), Android second (different mechanism needed — see below).

Track A — Apple (iosArm64, iosSimulatorArm64, ideally macosArm64):

  1. Fix the cinterop packaging gap first — this one matters beyond mobile.
    skainet_kernels.def declares no staticLibraries/libraryPaths; the archive
    is attached via binaries.all { linkerOpts(...) }, which does not travel to
    consumers. As published, the K/N klibs carry bindings but no machine code, so a
    downstream app can't link — this already affects the published -linuxarm64
    artifact today. Either embed staticLibraries in the .def or ship the archive
    in the klib. (Happy to split this into its own issue if you prefer.)
  2. Apple CMake invocations: -DCMAKE_SYSTEM_NAME=iOS against the iphoneos /
    iphonesimulator SDKs (+ macosArm64), one cmake-build-<target> dir and a
    configure/build/package task triple each, mirroring the existing -PcrossArm64
    pattern.
  3. Registration: K/N has no ServiceLoader; installNativeKernels() is a manual
    call, and PlatformCpuOpsFactory.apple.kt currently registers only the scalar
    provider. Needs a small design decision (documented startup call vs a hook), and
    ideally composes with the existing AccelerateCpuOps (vDSP dense FP32) so Apple
    gets NEON quant kernels and Accelerate FP32 rather than all-or-nothing.
  4. CI: Apple legs in publish.yml's build-native matrix and an
    iosSimulatorArm64Test lane (currently the only automatable way to exercise the
    iOS NEON path).

Track B — Android:

Real Android apps (ART/Compose) can't consume androidNativeArm64 klibs — that
target only serves pure Kotlin/Native binaries. The practical route is a small JNI
bridge: an AAR module with externalNativeBuild over the same C sources, jniLibs
for arm64-v8a, thin JNI shims per kernel, and a priority-100 KernelProvider in
androidMain registered from PlatformCpuOpsFactory.android.kt. It works against
the existing heap FloatArray/ByteArray kernel SPI, no storage redesign needed.
(If the K/N androidNativeArm64 route is wanted anyway, note
skainet-backend-api doesn't declare androidNative targets yet, while
skainet-lang-core already does.)

Small kernel gap for either track: q4_0_matmul.c is scalar-only (no NEON
body), while q8_0/q4k/q5k/q6k have one. Q4_0 is one of the priority mobile formats,
so it deserves a NEON body as part of this work.

Scale of the ask

At 1 tok/s on Android we need roughly 5x to be usable, not the full 21x the JVM
gets — even a partial result unblocks real apps. Related: #910 (vectorized eager
CPU kernels for K/N), #722 (accelerated CPU ops for linux native targets).

We have a physical-device test bench (Android phone + iPhone) and will gladly test
any branch or snapshot and report numbers back. Happy to contribute PRs for the
tracks above, starting with the cinterop packaging fix.

Ngôn ngữ chính
Kotlin
Star
52
Fork
15
Merge trung bình
1 ngày 8 giờ
Pull request đã merge (30 ngày)
37

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của SKaiNET-developers/SKaiNET

Tất cả issue của SKaiNET-developers/SKaiNET

Issue tương tự

Thêm issue về Kotlin

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.