Dense FP32 (and FP16) matmul on the Android JNI tier: attention runs scalar today
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
调研方向
Start by reading fp32_matmul.c, fp16_matmul.c, and the JNI exports in native/skainet_jni.c; then trace how JniKernelProvider registers kernels and review JniKernelParityTest and KernelSupportMatrixTest. Add the requested JNI matmul variants and registrations, verify scalar parity including offset and strided operands, regenerate the support matrix, and measure the whisper-tiny encoder on a device. Done means bit-identical attention-shape results and at least 5× speedup on a Pixel 7 Pro/8a-class phone.
由索引模型根据 Issue 内容生成。
描述
Context
A fully offline Android transcription app built on LiteRT (Whisper large-v3-turbo split encoder/decoder on OpenCL FP16, Parakeet TDT 0.6B, Silero VAD on ONNX Runtime) was evaluated as a SKaiNET consumer. Its author's own measurements on a Pixel 7 Pro are the bar: LiteRT CPU XNNPACK transcribes a 14 s memo in 82.2 s (RTF 5.9); the GPU path needs 34.4 s.
Since 0.50.0 the Android JNI tier (skainet-backend-jni-cpu) serves every GGML quant format from mapped weights on NEON, and 0.52.0 made the dispatch self-installing. That covers the weight side of a Whisper encoder (Q8_0 from whisper.cpp GGUFs, 874 MB). What it does not cover is the activation side.
Gap
The generated kernel support matrix (docs/.../reference/kernel-support-matrix.adoc, 0.54.0) lists Float32 and BFloat16 as scalar on Android, and native/skainet_jni.c exports only q40/q4k/q50/q51/q5k/q6k/q80 matmul entries plus the ternary gemv. A Whisper large-v3-turbo encoder issues, per layer, 20 heads of QKᵀ ([1500,64]×[64,1500]) and AV ([1500,1500]×[1500,64]) as FP32 activation × activation matmuls, 32 layers deep, plus two conv1d stem layers. On Android all of that runs through the scalar Kotlin path, so the quantized weight kernels cannot make the encoder fast on their own.
fp32_matmul.c already exists in skainet-backend-native-cpu (aarch64-verified, see #920) and the JVM reaches it through FFM; Android does not.
Scope
- JNI entries for dense FP32 matmul, including a batched variant and a transposed-right-operand variant (
A × Bᵀ) so attention does not pay a transpose copy per head, using the existingskainet_row_threadspool. - FP16 weight matmul on the JNI tier, since whisper.cpp ships F16 GGUFs and
fp16_matmul.cis in tree (#885 tracks the FFM side of the same kernel). -
JniKernelProviderregistrations soKernelDispatchselects them on Android at the native priority. - Parity tests against
ScalarFp32MatmulKernelinJniKernelParityTest, including offset/strided operands (the #1173 class of bug). -
KernelSupportMatrixTestregenerated:Float32andFloat16shownative-jnion Android. - Device measurement with the M2-A5 harness: whisper-tiny encoder before/after.
Acceptance
- Bit-identical results to the scalar path on device for the attention shapes above.
- whisper-tiny.en encoder on a Pixel 7 Pro / 8a class phone at least 5× faster than the scalar Android baseline.
Related
- #920 — NEON kernels to mobile (the JNI tier this extends)
- #885 — FP16 native kernel on the FFM tier
- #949 — the non-matmul Android overhead this issue does not address (see the companion issue on fused elementwise/norm kernels)
- 主要语言
- Kotlin
- 星标
- 52
- 派生
- 15
- 平均合并
- 1 天 15 小时
- 30 天内合并 PR
- 36
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
SKaiNET-developers/SKaiNET 的其他 Issue
-
coding good first issue size:xs skill:kotlin-core sub-issue
难度 2/5 1-3 小时 新手友好度 88/100
SKaiNET-developers/SKaiNET#1323 ·
维护者通常 1 天内回复
-
coding good first issue platform size:xs skill:js sub-issue
难度 2/5 1-3 小时 新手友好度 72/100
SKaiNET-developers/SKaiNET#1232 ·
维护者通常 1 天内回复
-
skeep tracking
难度 5/5 一周以上 新手友好度 15/100
SKaiNET-developers/SKaiNET#1331 ·
维护者通常 1 天内回复
-
assessment size:s skill:review sub-issue
难度 4/5 1-2 天 新手友好度 18/100
SKaiNET-developers/SKaiNET#1330 ·
维护者通常 1 天内回复
-
documentation good first issue size:s skill:docs sub-issue
难度 3/5 1-2 天 新手友好度 78/100
SKaiNET-developers/SKaiNET#1329 ·
维护者通常 1 天内回复
查看 SKaiNET-developers/SKaiNET 的全部 Issue
相似的 Issue
-
afk-ok area:data-quality importer size:S
难度 2/5 1-3 小时 新手友好度 77/100
enorm-labs/event-junkie#3166 ·
维护者通常 1 天内回复
-
bugbane/qf good first issue
难度 2/5 1-3 小时 新手友好度 64/100
osservatorionessuno/bugbane#173 ·
维护者通常 1 天内回复
-
Android: build fails with Android Gradle Plugin 9 (kotlinOptions, lintOptions, proguard-android.txt)未关闭
难度 2/5 1-3 小时 新手友好度 78/100
capacitor-community/stripe#544 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
-
documentation
难度 2/5 1-3 小时 新手友好度 66/100
theopenconversationkit/tock#2113 ·
维护者通常 1 天内回复