Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

EmbeddingService: use flutter_gemma batch API (generateEmbeddings) for sync-burst throughput

Abierto
  • ¿Algo no está bien?
#125 3 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

enhancement P0-Critical

Problem

EmbeddingService.embed (lib/features/shiv/rag/embedding/embedding_service.dart) calls EmbeddingModel.generateEmbedding(text) once per note. Every call
pays the full LiteRT per-invocation overhead (input/output tensor marshalling, GPU dispatch setup/teardown). The actual model math on Gecko-110M is
tiny — overhead dominates.

This is invisible when a user adds one note at a time, but becomes visible during sync bursts:

  • First app open after install / restore (relay backfills 100s of notes)
  • Returning online after a few days
  • Joining / catching up on a busy channel or long thread

In those windows the embedder lane saturates EmbeddingQueue(Semaphore(2)) for ~tens of seconds, RAG context lags behind the visible feed (Shiv answers
are "incomplete" until the index catches up), and the phone stays warm.

Proposal

Wire flutter_gemma's batch entry point through the service:

Future<List<List>> embedBatch(
List texts, {
bool isDocument = false,
});

Internally calls _model!.generateEmbeddings(texts, taskType: ...) — one native dispatch for the whole list — then L2-normalises each vector. Keep the
existing single-text embed() as a thin wrapper around embedBatch([text]).first so call sites that genuinely embed one string at a time don't change.

Call sites to migrate

  • The note-ingest path that embeds inbound Kind 1 events as the Gateway writes them to Isar. Currently iterates and calls embed(...) per note; should
    accumulate a small list (e.g. up to N=16 or whatever flushes a Gateway batch) and call embedBatch.
  • Saved-note / reference-note backfill jobs.

Out of scope — explicitly NOT changing

  • The embedding model itself (stay on bundled Gecko-1024 — offline-first, ~25 MB).
  • The 1-hop GraphRAG expansion. Edges come from e/t tags, not from the embedder.
  • The EmbeddingQueue(Semaphore(2)) parallelism cap — that stays as a backpressure guard.
  • Shiv chat query embedding — already single-string, no batching opportunity.

Expected user-visible improvements

Image

No change to single-add latency or retrieval quality.

Acceptance

  • New embedBatch(List) lands on EmbeddingService with GPU→CPU fallback preserved.
  • Inbound note handler in the Gateway batches embed calls.
  • Device test (one app install, integration_test/) ingests N≈100 notes and asserts wall-clock time drops by ≥5× vs. current per-note path on the same
    device.
  • No regression on chat first-token latency or single-note add.

References

Lenguaje dominante
Dart
Estrellas
2
Forks
2
Merge medio
1 d 7 h
PR fusionados (30 d)
35

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de basictech01/uniun

Todos los issues de basictech01/uniun

Issues similares

Más issues de Dart

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.