Incremental update (`graphrag update`) silently ignores removed documents; they stay indexed and retrievable

Aperta
  • Qualcosa non va?
#2,590 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
data

Direzione di ricerca

Start with graphrag/index/update/incremental_index.py and graphrag/index/workflows/load_update_documents.py, then trace how deleted inputs are handled by the update pipeline. Check the update workflow tests for coverage of removal-only changes. Done means removed documents and their derived data are no longer retrievable, or the update clearly warns or fails if removals require a full re-index.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

In GraphRAG 3.2.0, the incremental update computes which documents were removed from
the input, but never uses that information. The update completes without error, and the
removed documents, together with their text units, entities, relationships and community
reports, stay in the index and remain retrievable by queries.

This was first reported to MSRC (VULN-235383), which confirmed the behavior and classified
it as a functional and data-lifecycle defect rather than a security vulnerability, and
suggested filing it here.

Root cause

  • graphrag/index/update/incremental_index.py: get_delta_docs() returns an InputDelta
    with both new_inputs and deleted_inputs.
  • graphrag/index/workflows/load_update_documents.py: load_update_documents() returns only
    delta_documents.new_inputs; deleted_inputs is not referenced anywhere else in the package.
  • When the only change is a removal, the workflow logs No new update documents found. and
    stops the pipeline normally, so every output table is left unchanged.

Two related effects:

  • Each update writes a copy of the previous index, including the removed documents, to
    update_output/<timestamp>/previous/. A later full re-index does not touch this directory.
  • The LLM cache keeps the responses derived from the removed documents, also after a full re-index.
Steps to reproduce
  1. pip install graphrag==3.2.0, then graphrag init --root ./proj and configure a model.
  2. Put a.txt and b.txt into ./proj/input, where b.txt contains a unique fact,
    e.g. Alice Costa's badge number is Q7F9K2P1XZ3A.
  3. graphrag index --root ./proj
  4. Delete ./proj/input/b.txt and run graphrag update --root ./proj.
    The run succeeds; the log shows No new update documents found.
  5. ./proj/output/documents.parquet still lists b.txt, text_units.parquet still contains
    the badge number, and
    graphrag query --root ./proj --method local --query "What is Alice Costa's badge number?"
    still returns it.
  6. ./proj/update_output/<timestamp>/previous/*.parquet and ./proj/cache/ also contain
    b.txt's content.

In a larger experiment (10 synthetic people, 20 background people, Qwen2.5-7B-Instruct via
vLLM, local search), all removed documents stayed indexed after the incremental update, and
local search recovered 77% of the removed people's unique facts, the same as before the
removal. After a full re-index, local search recovered none of them, but the LLM cache and
the update backups still contained them.

Expected Behavior

Either

  • the incremental update processes deleted_inputs: it removes those documents and their
    text units and recomputes the affected entities, relationships and communities; or
  • at minimum, it warns or fails when the input has lost documents, and the documentation
    states that removing documents requires a full re-index.

It would also help to prune, or document, the update backups and the cache entries that are
derived from removed documents. Operators commonly remove documents to honour data-erasure
requests or to take out incorrect or poisoned content, and a silent success makes them
believe the content is gone.

GraphRAG Config Used

Default settings.yaml from graphrag init, with the completion and embedding models
pointed at local OpenAI-compatible endpoints. The behavior is in the update workflow and
does not depend on the model.

Additional Information
Lingua principale
Python
Stelle
36.6k
Fork
3.9k
Merge medio
4g 11h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/graphrag

Tutte le issue di microsoft/graphrag

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.