Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Undocumented 1024-dim embedding floor fails asynchronously while /health still reports embed: true

Chiusa
  • Qualcosa non va?
#434 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@gloryfromca ci sta già lavorando.

Dal 16/10/2026.

  • #510 di @gloryfromca — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend-api-design

Direzione di ricerca

Start by tracing how the [embedding] configuration is checked at startup and how /api/v2/health decides to report embed: true; the issue does not name source files or tests. Reproduce with the provided 768-dimension Ollama-compatible configuration, then follow the POST /api/v2/memory/add cascade and inspect the resulting retryable status. Done when a dimension mismatch is surfaced before data is silently lost, and the health result no longer reports a working embedder for that mismatch.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

[embedding] hard-requires vectors of at least 1024 dimensions, but this floor is
undocumented and unvalidated at startup. A 768-dim provider is accepted everywhere it is
checked, and only fails later, asynchronously, in the ingest cascade — while /health
continues to report embed: true and ok.

The practical result is an index that silently contains nothing and a health endpoint that
says it is fine.

Reproduce

Configure any 768-dim embedding provider — Ollama's nomic-embed-text is the obvious one,
since it is the most common local default:

[embedding]
provider = "openai"          # Ollama's OpenAI-compatible endpoint
base_url = "http://127.0.0.1:11434/v1"
model    = "nomic-embed-text"

Then:

  1. everos server start
  2. GET /api/v2/health → {"embed": true, "status": "ok"} ← passes
  3. POST /api/v2/memory/add → 202 Accepted ← accepted
  4. The background cascade fails:
ValidationError: List should have at least 1024 items after validation, not 768
  1. Rows are persisted with retryable=FALSE.
  2. POST /api/v2/memory/search → zero results, HTTP 200.
  3. GET /api/v2/health → still ok, still embed: true.
Why this is worth fixing

Every surface a user would check to answer "is my embedding provider working?" returns a
positive answer:

  • the health endpoint reports embed: true
  • the write returns 202
  • search returns 200, just with an empty result list

Nothing distinguishes "the corpus contains no match" from "nothing was ever indexed."
Silent, non-retryable data loss with a green health check is a bad failure mode for a
memory system specifically, because the symptom — recall that comes back empty — is
indistinguishable from normal operation on a cold store.

Suggested fix

Any one of these would be enough; the first is the cheapest:

  1. Validate at startup. Embed a probe string once when the provider is configured,
    compare len(vector) against _DEFAULT_DIM, and refuse to start (or log a hard error)
    on mismatch. This turns a silent async failure into a loud synchronous one.
  2. Reflect it in /health. embed: true should mean "an embedding of the correct
    dimension was produced", not "a provider is configured".
  3. Document the floor. At minimum, state the ≥1024 requirement next to the
    [embedding] config block, and name a couple of local models that satisfy it —
    this is not discoverable from the config reference today.
Workaround

Use a provider whose native dimension is ≥1024, or truncate client-side. For Ollama,
qwen3-embedding:8b (4096-dim) works; nomic-embed-text (768) does not.

Environment
  • everos 1.2.3, Python 3.12, Linux
  • LanceDB backend, local Ollama for both LLM and embeddings
  • Found while ingesting a 45-file Markdown corpus; 0 of the first batch became searchable
    before the dimension mismatch was identified
Lingua principale
Python
Stelle
13.3k
Fork
926
Merge medio
17h 14m
PR unite (30g)
40

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di EverMind-AI/EverOS

Tutte le issue di EverMind-AI/EverOS

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.