Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Wiping .index/lancedb leaves an empty index that every health surface reports as healthy

Cerrado
  • ¿Algo no está bien?
#501 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Ya se ha fusionado un pull request relacionado.

  • #522 de @gloryfromca — fusionado

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python, sqlite
Área
databases, observability

Línea de trabajo

Start with reconcile_index_once() and the .index_reconcile_version marker, which the issue says sits in .index/ rather than .index/lancedb/, then trace how cascade status and the GET /health probe read the sqlite ledger. Done means that after rm -rf .index/lancedb, the system reports the mismatch (a per-table row count beside the done count) instead of healthy, and a test covers that case. The issue leaves open whether to refuse startup, enqueue a rebuild, or move the marker, so settle that choice with the maintainers before writing code.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Summary

Deleting ~/.everos/.index/lancedb leaves a store that contains nothing and reports perfect health: cascade status shows every file done with lag: 0, GET /health reports cascade.healthy: true with zero failures, and search returns 0 hits against an empty table. Markdown is untouched, so nothing is lost — but nothing says anything is wrong either.

This is a detectability gap, not an undocumented trap. The hazard itself is documented, clearly and in more than one place (see Already documented below). What is missing is that the running system can tell this state apart from a healthy one with a two-number comparison, and doesn't.

Found while reproducing #334 on current main; the rm -rf came from that report's steps.

Reproduce

EverOS 1.4.1 at a345d08, isolated memory root, real everos server start, DeepInfra Qwen3-Embedding-4B. Corpus built with EverOS's own EpisodeWriter: 106 daily-log files, 1060 entries.

  1. Start the server, let the cascade index everything (cascade status → done: 106, lag: 0).
  2. Stop the server.
  3. rm -rf <root>/.index/lancedb — only lancedb, leaving .index/sqlite and the rest of .index/ in place.
  4. Start the server again.

Observed

surface reports
everos cascade status pending 0 · done 106 · last_processed 106 · lag 0
GET /health status: ok · cascade.healthy: true · failed_permanent 0 · failed_retryable 0
LanceDB rows episode 0 · atomic_fact 0 · user_profile 0
POST /api/v2/memory/search 0 hits (HTTP 200)
Markdown source of truth 106 files, intact

Mechanism

  1. The cascade ledger survives the wipe. It lives in .index/sqlite/system.db, so every md row is still status='done' and the scanner enqueues nothing.
  2. The self-heal marker survives too. reconcile_index_once() force-enqueues every indexed md once per INDEX_RECONCILE_VERSION, recording that in <index_dir>/.index_reconcile_version. That file sits in .index/, not inside .index/lancedb/, so wiping the backend leaves it holding the current version and the pass is a no-op.
  3. Nothing compares the ledger against the tables. "Every row done" and "the tables are empty" are never checked against each other, at startup or in the health probe.

Already documented — scope this accordingly

An earlier revision of this issue claimed the hazard was "only visible to someone who already thought to read the rebuild command's help". That was wrong, and it overstated the problem. The project documents it properly:

  • docs/cascade_runbook.md has a "Why not a bare rm" comparison table — rm -rf .index/lancedb → "❌ scanner skips done rows → empty index" — and a bolded paragraph repeating it in prose.
  • docs/how-memory-works.md carries the same warning.
  • The everos cascade rebuild docstring states it.
  • The LanceDBSchemaMismatchError message now names everos cascade rebuild. The 1.1.2-era text that pointed users at rm -rf ~/.everos/.index/lancedb (quoted in #334) is gone.

So the product no longer walks anyone into this, and an operator who reads the runbook will not hit it. Severity should be read as hardening, not as an outage: reaching this state takes a deliberate rm -rf against an internal directory that the documentation tells you not to delete.

What is still worth fixing

The detection, not the documentation. The state is trivially distinguishable and the system reports it as healthy:

  • Startup check. When the ledger says a kind is fully done but that kind's table holds zero rows, something external removed the index — log an error and either refuse to start or enqueue a rebuild. A row count per table is O(1) against LanceDB metadata.
  • Move the marker. Put .index_reconcile_version inside .index/lancedb/ so wiping the backend also invalidates the marker that governs re-indexing it. The reconcile pass then self-arms after exactly the operation it should fire for, and this specific state stops existing.
  • Surface the comparison. A rows count per table beside the queue counters in GET /health and cascade status. An operator seeing done: 106 next to episode rows: 0 needs no runbook.

Recovery (verified)

everos cascade rebuild restores it — 1060 rows back from the same 106 Markdown files, measured. No memory content is lost.

Related: #334 (origin of the rm -rf), and the same "the system cannot tell a broken index from an empty corpus" shape as #434 and #364.

Lenguaje dominante
Python
Estrellas
13.3k
Forks
926
Merge medio
22 h 56 min
PR fusionados (30 d)
27

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de EverMind-AI/EverOS

Todos los issues de EverMind-AI/EverOS

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.