Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

EmbeddingService: use flutter_gemma batch API (generateEmbeddings) for sync-burst throughput

未关闭
  • 哪里不对？
#125 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

这个 Issue 还没有评估数据。

描述

enhancement P0-Critical

Problem

EmbeddingService.embed (lib/features/shiv/rag/embedding/embedding_service.dart) calls EmbeddingModel.generateEmbedding(text) once per note. Every call
pays the full LiteRT per-invocation overhead (input/output tensor marshalling, GPU dispatch setup/teardown). The actual model math on Gecko-110M is
tiny — overhead dominates.

This is invisible when a user adds one note at a time, but becomes visible during sync bursts:

  • First app open after install / restore (relay backfills 100s of notes)
  • Returning online after a few days
  • Joining / catching up on a busy channel or long thread

In those windows the embedder lane saturates EmbeddingQueue(Semaphore(2)) for ~tens of seconds, RAG context lags behind the visible feed (Shiv answers
are "incomplete" until the index catches up), and the phone stays warm.

Proposal

Wire flutter_gemma's batch entry point through the service:

Future<List<List>> embedBatch(
List texts, {
bool isDocument = false,
});

Internally calls _model!.generateEmbeddings(texts, taskType: ...) — one native dispatch for the whole list — then L2-normalises each vector. Keep the
existing single-text embed() as a thin wrapper around embedBatch([text]).first so call sites that genuinely embed one string at a time don't change.

Call sites to migrate

  • The note-ingest path that embeds inbound Kind 1 events as the Gateway writes them to Isar. Currently iterates and calls embed(...) per note; should
    accumulate a small list (e.g. up to N=16 or whatever flushes a Gateway batch) and call embedBatch.
  • Saved-note / reference-note backfill jobs.

Out of scope — explicitly NOT changing

  • The embedding model itself (stay on bundled Gecko-1024 — offline-first, ~25 MB).
  • The 1-hop GraphRAG expansion. Edges come from e/t tags, not from the embedder.
  • The EmbeddingQueue(Semaphore(2)) parallelism cap — that stays as a backpressure guard.
  • Shiv chat query embedding — already single-string, no batching opportunity.

Expected user-visible improvements

Image

No change to single-add latency or retrieval quality.

Acceptance

  • New embedBatch(List) lands on EmbeddingService with GPU→CPU fallback preserved.
  • Inbound note handler in the Gateway batches embed calls.
  • Device test (one app install, integration_test/) ingests N≈100 notes and asserts wall-clock time drops by ≥5× vs. current per-note path on the same
    device.
  • No regression on chat first-token latency or single-note add.

References

主要语言
Dart
星标
2
派生
2
平均合并
19 小时 21 分钟
30 天内合并 PR
37

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

basictech01/uniun 的其他 Issue

查看 basictech01/uniun 的全部 Issue

相似的 Issue

更多 Dart Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。