Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

feat: fast on-device note translation with ML Kit, keeping the LLM as fallback

Đang mở
  • Có gì không ổn?
#241 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
25/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
dart, flutter
Lĩnh vực
machine-learning, mobile

Hướng nghiên cứu

Start by measuring cold and warm translation on a device as described in “Measure on a device,” then read the existing TranslateNoteUseCase → TranslationPrompt → GenerateOneShotUseCase path. Compare ML Kit and the current LLM on real notes, including language detection, downloads, storage, and protected text. Done means the supported-language path is fast, fallback cases still use the LLM, tests cover the choice, and a release APK builds.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement P2-Medium

🗂️ Backlog — not scheduled yet. Before building: connect a phone and measure speed first (section "Measure on a device"), and look for something better than the options below.

Translating a note today runs a one-shot prompt through the on-device LLM (TranslateNoteUseCase → TranslationPrompt → GenerateOneShotUseCase). It is slow: the audit measured prompt prefill at 19.9 s on CPU and ~7.9 s on GPU on a mid-range phone, before the first translated word appears — and it needs a chat model downloaded at all, competing with chat and background AI work for the same device.

ML Kit's on-device translation (google_mlkit_translation) is a dedicated translation model — "the same models used by the Google Translate app's offline mode" — and typically answers in well under a second once its language model is on the device.

Proposal: ML Kit first, LLM as the fallback

Keep the existing LLM path; put ML Kit in front of it where it applies.

  • Supported pairs → ML Kit. Instant, offline after the first download, no LLM.
  • Everything else → the current LLM path, unchanged.

What ML Kit covers — checked against our 22 languages

TranslationLanguage.all offers 22 languages. ML Kit supports 20 of them. Not supported: Malayalam (ml) and Punjabi (pa) — those must stay on the LLM path.

Caveats found while checking — each shapes the design

  • Models are downloaded, not bundled — ~30 MB per language, fetched from Google on first use, then fully offline. The first translation into a language needs a network and a visible download state (and probably a Wi-Fi-only option). Downloaded models should be listed/removable, like the AI models are today.
  • Everything pivots through English. Google: models are "trained to translate to and from English. When you translate between non-English languages, English is used as an intermediate translation, which can affect quality." Hindi → Tamil goes Hindi → English → Tamil.
  • Quality is stated as "casual and simple translations." Fine for reading a note; worth a manual comparison against the LLM on real notes before making it the default.
  • Code-mixed / romanised text (Hinglish). The current prompt handles it explicitly ("translate the meaning, not the spelling"). ML Kit expects a known source language in its native script. Route romanised or undetected input to the LLM.
  • The source language must be known. The LLM infers it; ML Kit needs it. Likely google_mlkit_language_id (on-device) — check its size and native dependencies before adding it.
  • Build dependencies checked: the Android com.google.mlkit:translate artifact brings no Firebase services (only firebase-components/firebase-encoders, already in the app through mobile_scanner), but it does pull in okhttp 3.0.0 for model downloads — check it resolves cleanly alongside anything else on the classpath. After adding it, build a release APK: ML Kit plugins have broken R8 before (#240's work).
  • Use the specific google_mlkit_translation package, not the google_ml_kit umbrella — the umbrella pulls in 21 ML Kit plugins.

Done when

Translating a note into a supported language uses ML Kit and shows the result in about a second once the model is present; the first use of a language shows a download state; Malayalam, Punjabi, romanised and undetectable text still translate through the LLM; a release APK builds; and the choice is covered by tests with the ML Kit side behind a fake, as OCR and labeling are.

Storage — download only what is used

Models are per language, downloaded on first use, deletable. Nobody stores all 20:

User Languages used ML Kit storage
Reads Hindi notes, translates to English Hindi (English may be built in — unverified) ~30–60 MB
Hindi + Tamil + English 2–3 ~60–90 MB
All 20 languages 20 ~600 MB — unrealistic, and deletable

Design to keep it small:

  1. Download a language only when the user first translates into it — never all up front.
  2. List downloaded languages in Settings with sizes and a delete button, like the AI models today.
  3. Keep the AI model as the zero-storage option — no download needed, just slower.
  4. Ask before downloading on mobile data (Wi-Fi only by default).

Other offline options researched

Option Our 22 languages Size Licence Flutter effort Verdict
ML Kit 20 (no Malayalam, Punjabi) ~30 MB per language free to use (closed Google models) existing plugin — a day or two use now, AI as fallback
Mozilla Bergamot (Firefox's offline translator) 21 (only Punjabi missing) 17–44 MB per direction MPL-2.0 no Flutter package — build the C++ engine for Android + iOS ourselves best single engine if native work is taken on later
IndicTrans2 (AI4Bharat) 11 — all 10 Indian languages + English ~260–380 MB per direction, but one model covers all Indian languages MIT heavy — custom ONNX + porting its preprocessing to Dart best Indian-language quality (on par with Google Translate)
NLLB-200 (Meta) 21+ ~2.5 GB non-commercial weights — ❌ cannot ship
TranslateGemma 4B 22 2–4 GB Gemma terms already supported ❌ slower than today (CPU-only, ~90 s before output)
OPUS-MT / Apple Translation too few Indian languages — — — ❌
AI model already on the phone (today) 22 0 MB extra — — slow: 7.9 s (GPU) – 19.9 s (CPU) before the first word

Nothing offline is meaningfully smaller at usable quality: ~17–30 MB per language/direction is the floor for real translation models.

Implementation notes from the research

  • ML Kit needs the source language. Add google_mlkit_language_id. It also catches Hinglish: Latin-script text identified as Hindi goes to the AI.
  • Romanised / code-mixed text is weak on every translation engine — all are trained on native script. Keep the AI path for it.
  • Protect hashtags, @mentions, URLs, emoji and line breaks. The AI prompt tells the model to keep them; a translation engine has no such instruction, so mask them before translating and restore after.
  • ML Kit and Bergamot pivot through English for Indian → Indian pairs; only IndicTrans2 translates between Indian languages directly.

Measure on a device before building

  • Speed, cold and warm, for English→Hindi, Hindi→English, Hindi→Tamil (pivoted) and English→Malayalam — against today's 7.9–19.9 s.
  • Language detection on one-sentence notes and Hinglish.
  • A native-speaker check of ~30 real notes per language, including hashtags, mentions and links.
  • The model download on mobile data, and storage as languages are added.

Sources

ML Kit languages · ML Kit translation · Mozilla translations · Bergamot on Android · IndicTrans2 · IndicTrans2 paper · NLLB-200 licence · TranslateGemma

Ngôn ngữ chính
Dart
Star
2
Fork
2
Merge trung bình
18 giờ 53 phút
Pull request đã merge (30 ngày)
35

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của basictech01/uniun

Tất cả issue của basictech01/uniun

Issue tương tự

Thêm issue về Dart

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.