contradiction="llm" raises a raw pydantic error after the insert is written

Aperta
#336 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python

Direzione di ricerca

Read src/selfgraph/contradiction.py at judge_contradictions and compare its verdict handling with extraction's error handling. Reproduce the malformed reply using the provided embedded Kuzu example, then inspect the existing tests for contradiction judging and partial failures. Done means malformed or client-error replies are handled consistently and do not leave the newly inserted relation and source in an unintended current state.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

approved-for-pr area: errors area: maintenance bug effort: S P2

What happens

With contradiction="llm", the judge's reply is checked with _Verdict.model_validate(verdict) and nothing catches the result. A malformed reply raises a raw pydantic_core.ValidationError out of insert.facts, with no code or hint. The judge runs after upsert_relations, so on an auto-commit backend the new relation and its source are already written: both relations stay current and no judgment is applied.

Other replies that do this:

  • None, "[1]", {"retired": None}, {"retired": ["x"]}.
  • An object with a retired attribute (extraction accepts attribute objects; the judge does not).
  • An exception raised by the client itself. Extraction turns that into extraction_partial_failure; the judge lets it escape.
  • A _Verdict.model_construct(retired=["0"]) reply skips validation and fails with TypeError at the 1 <= number comparison.

Lax coercion also means {"retired": [True]} and {"retired": ["1"]} retire fact 1.

Reproduce

On main (5a277c9), embedded Kuzu:

import asyncio, tempfile
from selfgraph import KuzuStore, SelfGraph

class LLM:
    async def generate(self, *, system, user, output):
        return {"retired": None} if output.__name__ == "_Verdict" else output()

SCHEMA = {"entities": [{"name": "Person"}, {"name": "Company"}], "relations": [{"type": "WORKS_AT"}]}

async def main():
    with tempfile.TemporaryDirectory() as tmp:
        async with SelfGraph(store=KuzuStore(f"{tmp}/g.kuzu"), llm=LLM(), schema=SCHEMA, contradiction="llm") as g:
            await g.insert.facts({
                "entities": [{"type": "Person", "name": "Ann"}, {"type": "Company", "name": "Acme"},
                             {"type": "Company", "name": "Globex"}],
                "relations": [{"from_entity": "person:ann", "type": "WORKS_AT", "to_entity": "company:acme",
                               "event_time": "2020-01-01"}],
            })
            try:
                await g.insert.facts({"relations": [{"from_entity": "person:ann", "type": "WORKS_AT",
                                                     "to_entity": "company:globex", "event_time": "2021-01-01"}]},
                                     source_id="doc2")
            except Exception as exc:
                print(type(exc).__module__, type(exc).__name__)
            rels = (await g.relations.list(include_invalidated=True)).items
            print([(r.to_entity, r.is_current, r.source_count) for r in rels])

asyncio.run(main())

Output:

pydantic_core._pydantic_core ValidationError
[('company:acme', True, 0), ('company:globex', True, 1)]

Cause

src/selfgraph/contradiction.py, judge_contradictions: verdict = verdict if isinstance(verdict, _Verdict) else _Verdict.model_validate(verdict), with no error mapping, no from_attributes, no handling of client exceptions, and an isinstance shortcut that trusts unvalidated instances.

Lingua principale
TypeScript
Stelle
173
Fork
58
Merge medio
6h
PR unite (30g)
120

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di hydra-db/selfgraph

Tutte le issue di hydra-db/selfgraph

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.