Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Tracking: Whisper-class offline ASR on Android — engine-side P0

Aperta
  • Qualcosa non va?
#1,282 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
android, kotlin
Ambito
machine-learning, mobile

Direzione di ricerca

Start with the linked P0 issues #1275–#1281 to identify a bounded engine-side task; this tracking issue spans native kernels, scheduling, VAD, ONNX import, and numerical tooling. Use the stated Pixel-class whisper-tiny.en exit criterion and M2-A5 harness as the completion guide: the encoder must be at least 5× faster than the scalar path, matmuls must name their kernels, and Silero VAD must match ONNX Runtime within 1e-4 without an ORT dependency.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

compute-backend skill:android tracking

Goal

Make SKaiNET the obvious engine for a fully offline, Whisper-class transcription app on Android. The reference case is an existing app on LiteRT (Whisper large-v3-turbo split encoder/decoder on OpenCL FP16, Parakeet TDT 0.6B, Silero VAD on ONNX Runtime, a 350M LLM for titles on LiteRT-LM): three native runtimes in one APK, hand-written mel/FFT/resampler/tokenizers/decode loops, and model-file surgery plus byte-patched runtime AARs to work around GPU numerics bugs. Its author's Pixel 7 Pro numbers are the bar:

Path 14 s memo RTF
LiteRT CPU, XNNPACK, 2×3 threads 82.2 s 5.9
LiteRT GPU, OpenCL FP16 34.4 s 2.5

Target for this milestone: CPU-only SKaiNET Whisper large-v3-turbo at or below 82 s on the same class of device, token-identical to whisper.cpp, with the audio side and VAD served by SKaiNET libraries.

What 0.54.0 already provides

  • JNI NEON backend (skainet-backend-jni-cpu) for every GGML quant format, runtime dotprod dispatch, threaded packed matmuls, self-installing dispatch (#920 → 0.50–0.52).
  • Off-heap mapped weights on Android (AndroidGguf.loader, WeightForm(residency = MAPPED)), memory plan and device fit check before load; a 1.0 GB Q4_K_M LLM decodes at 61–66 ms/step on a Pixel 8a.
  • Lstm with explicit state (#823); transformer-core KV cache; tape → StableHLO export at billion-parameter scale (0.53.0).

Engine-side P0 (this repo)

  • #1275 — dense FP32/FP16 matmul on the JNI tier (attention is scalar on Android today)
  • #1276 — native fused LayerNorm / GELU / softmax / conv1d kernels
  • #1277 — Schedule on Android verified and measured for SDPA
  • #1278 — native Silero VAD model on Lstm, weights from the upstream ONNX file
  • #1279 — ONNX importer: LSTM, Gather, Unsqueeze/Squeeze, Cast, Where, If
  • #1280 — TensorDump + TensorCompare for divergence localisation
  • #1281 — FP16 simulation context + FP32-accumulating LayerNorm lowering rule

Out of this repo, tracked with the ASR projects

  • Whisper v3 128-mel MelConfig in the audio library (HF-exact constants).
  • Whisper large-v3-turbo model + self-attention KV cache + timestamp rules + long-form windowing in the Whisper project, on top of the current engine pin.
  • Consumer docs: the artifact picker must list skainet-backend-jni-cpu; the AndroidGguf KDoc still says packed tensors heap-stage (fixed in 0.50.0).

Follow-ups already identified (P1, separate issues when P0 lands)

  • AndroidGguf.loader over an AssetFileDescriptor so a noCompress GGUF is mapped straight from the APK instead of copied out.
  • Windowed mapping to lift the 2 GB single-region cap.
  • HF tokenizer.json byte-level BPE loader for Whisper vocab.
  • Parakeet TDT cartridge reusing Lstm and the 128-mel config.

Exit criterion

whisper-tiny.en runs on a Pixel-class phone through the JNI tier with mapped weights and the M2-A5 harness names the kernel behind every matmul; the encoder is ≥ 5× faster than the 0.54.0 scalar path; Silero VAD matches ONNX Runtime within 1e-4 with no ORT dependency.

Lingua principale
Kotlin
Stelle
52
Fork
15
Merge medio
1g 8h
PR unite (30g)
37

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di SKaiNET-developers/SKaiNET

Tutte le issue di SKaiNET-developers/SKaiNET

Issue simili

Altre issue su Kotlin

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.