Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Tracking: Whisper-class offline ASR on Android — engine-side P0

オープン
  • 何か問題がありますか？
#1,282 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
20/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
android, kotlin
領域
machine-learning, mobile

調査の方向性

Start with the linked P0 issues #1275–#1281 to identify a bounded engine-side task; this tracking issue spans native kernels, scheduling, VAD, ONNX import, and numerical tooling. Use the stated Pixel-class whisper-tiny.en exit criterion and M2-A5 harness as the completion guide: the encoder must be at least 5× faster than the scalar path, matmuls must name their kernels, and Silero VAD must match ONNX Runtime within 1e-4 without an ORT dependency.

索引モデルが issue の本文から書いたものです。

説明

compute-backend skill:android tracking

Goal

Make SKaiNET the obvious engine for a fully offline, Whisper-class transcription app on Android. The reference case is an existing app on LiteRT (Whisper large-v3-turbo split encoder/decoder on OpenCL FP16, Parakeet TDT 0.6B, Silero VAD on ONNX Runtime, a 350M LLM for titles on LiteRT-LM): three native runtimes in one APK, hand-written mel/FFT/resampler/tokenizers/decode loops, and model-file surgery plus byte-patched runtime AARs to work around GPU numerics bugs. Its author's Pixel 7 Pro numbers are the bar:

Path 14 s memo RTF
LiteRT CPU, XNNPACK, 2×3 threads 82.2 s 5.9
LiteRT GPU, OpenCL FP16 34.4 s 2.5

Target for this milestone: CPU-only SKaiNET Whisper large-v3-turbo at or below 82 s on the same class of device, token-identical to whisper.cpp, with the audio side and VAD served by SKaiNET libraries.

What 0.54.0 already provides

  • JNI NEON backend (skainet-backend-jni-cpu) for every GGML quant format, runtime dotprod dispatch, threaded packed matmuls, self-installing dispatch (#920 → 0.50–0.52).
  • Off-heap mapped weights on Android (AndroidGguf.loader, WeightForm(residency = MAPPED)), memory plan and device fit check before load; a 1.0 GB Q4_K_M LLM decodes at 61–66 ms/step on a Pixel 8a.
  • Lstm with explicit state (#823); transformer-core KV cache; tape → StableHLO export at billion-parameter scale (0.53.0).

Engine-side P0 (this repo)

  • #1275 — dense FP32/FP16 matmul on the JNI tier (attention is scalar on Android today)
  • #1276 — native fused LayerNorm / GELU / softmax / conv1d kernels
  • #1277 — Schedule on Android verified and measured for SDPA
  • #1278 — native Silero VAD model on Lstm, weights from the upstream ONNX file
  • #1279 — ONNX importer: LSTM, Gather, Unsqueeze/Squeeze, Cast, Where, If
  • #1280 — TensorDump + TensorCompare for divergence localisation
  • #1281 — FP16 simulation context + FP32-accumulating LayerNorm lowering rule

Out of this repo, tracked with the ASR projects

  • Whisper v3 128-mel MelConfig in the audio library (HF-exact constants).
  • Whisper large-v3-turbo model + self-attention KV cache + timestamp rules + long-form windowing in the Whisper project, on top of the current engine pin.
  • Consumer docs: the artifact picker must list skainet-backend-jni-cpu; the AndroidGguf KDoc still says packed tensors heap-stage (fixed in 0.50.0).

Follow-ups already identified (P1, separate issues when P0 lands)

  • AndroidGguf.loader over an AssetFileDescriptor so a noCompress GGUF is mapped straight from the APK instead of copied out.
  • Windowed mapping to lift the 2 GB single-region cap.
  • HF tokenizer.json byte-level BPE loader for Whisper vocab.
  • Parakeet TDT cartridge reusing Lstm and the 128-mel config.

Exit criterion

whisper-tiny.en runs on a Pixel-class phone through the JNI tier with mapped weights and the M2-A5 harness names the kernel behind every matmul; the encoder is ≥ 5× faster than the 0.54.0 scalar path; Silero VAD matches ONNX Runtime within 1e-4 with no ORT dependency.

主要言語
Kotlin
スター
52
フォーク
15
平均マージ
1日 15時間
マージ済み PR（30日）
36

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

SKaiNET-developers/SKaiNET のほかの issue

SKaiNET-developers/SKaiNET の issue をすべて見る

似ている issue

Kotlin の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。