Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Spike: llama.rn inference on the Note 9S

Aperta
  • Qualcosa non va?
#64 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
android, react-native, typescript
Ambito
machine-learning, mobile-dev, performance

Direzione di ricerca

Resolve dependency #63 first, then run the minimal Expo SDK 57 dev build with llama.rn on the Redmi Note 9S. Test the listed GGUF sizes, quantizations, thread counts, prompt shapes, and JSON-schema generation, recording device details and measurements in evidence/64/bench.json. Finish with the decision in docs/adr/0001-on-device-runtime.md and a screen recording.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area:mobile needs-device type:spike

Part of the JobTrail v2 rebuild · design doc · all drafts · draft #3

Depends on: #63

Context. Decides which model sizes and quantizations are viable on the minimum-spec device, before we train anything. Legacy never ran a model on a device.

Scope

  • Minimal Expo SDK 57 dev build (New Architecture) with llama.rn; Android build on Windows (local Android SDK or EAS).

  • Load off-the-shelf GGUFs: Gemma 3 270M and Qwen3-0.6B at Q8_0, Q4_K_M, Q4_0 (+ one ~1B model if RAM allows).

  • Measure per config: model load time, TTFT, prefill tok/s, decode tok/s, peak RSS, with threads = 2 / 4 / 6.

  • Measure two prompt shapes: long (~400-token instructions + schema + note) vs short (note only, ~100 tokens) — quantifies the prefill cost fine-tuning would remove.

  • Test JSON-schema-constrained generation works (grammar) and its overhead.

  • Record Note 9S RAM variant and Android version.

Acceptance criteria

  • evidence/64/bench.json with all measurements + device info; screen recording of one run.

  • Short written decision (ADR docs/adr/0001-on-device-runtime.md): viable size range, best thread count, quant formats to carry forward, p50 latency target for the product.

Definition of done (all issues)

Definition of done (every issue):

  • Acceptance criteria met with evidence linked in the closing comment (test run, results JSON, screenshot/recording).

  • needs-device issues: benchmark JSON + short screen recording from the named device committed under evidence/<issue#>/.

  • No claim in README/docs that isn't backed by a committed artifact.

  • CI green.

Device shorthand: Note 9S = Redmi Note 9S 6 GB (Snapdragon 720G, CPU-only), the minimum-spec device. iPhone = iPhone 16 Pro.

Budget rule: free tiers first (Gemini API free tier, Kaggle/Colab free GPUs, HF Hub, Vercel Hobby). Any script calling a cloud model must throttle to free-tier limits, checkpoint/resume, and log token usage. Paid spend only with owner approval, capped at a few dollars.

Lingua principale
Python
Stelle
0
Fork
0
Merge medio
14h 46m
PR unite (30g)
21

Preparare l'ambiente

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Taleef7/JobTrail

Tutte le issue di Taleef7/JobTrail

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.