replace_source retires a source's facts when one LLM call fails

オープン
#356 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
55/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
python, typescript
領域
ai, databases

調査の方向性

The issue identifies Insert.text, _apply_facts, and _ops.replace_source as the relevant entry points; inspect how extraction partial failures are represented and handled in both SDKs. Reproduce the supplied Kuzu scenario and trace how failed calls affect retirement. Done when a failed LLM call during replacement does not silently retire facts, with the behavior covered in both SDKs.

索引モデルが issue の本文から書いたものです。

説明

area: extraction area: write bug effort: M P1

What happens

insert.text(..., source_id=..., replace_source=True) retires every relation the earlier versions of the source stated and the new text "no longer yields". When one LLM call fails on a chunk (a timeout, a rate limit, a provider error), extraction carries on with an extraction_partial_failure warning, and the relations that call would have returned count as no longer stated. They are retired with fact_retired_by_source.

So a transient provider error during a re-ingest silently removes valid facts from the current graph. Both SDKs do this.

Reproduce

On main (5a277c9), embedded Kuzu:

import asyncio, tempfile
from selfgraph import KuzuStore, SelfGraph

REPLY = {
    "entities": [{"type": "Person", "name": "Ann"}, {"type": "Company", "name": "Acme"}],
    "relations": [{"from_name": "Ann", "type": "WORKS_AT", "to_name": "Acme"}],
}

class LLM:
    fail = False

    async def generate(self, *, system, user, output):
        kind = next(iter(output.model_fields))
        if self.fail and kind == "relations":
            raise TimeoutError("provider timed out")
        return {kind: REPLY.get(kind, [])}

async def main():
    with tempfile.TemporaryDirectory() as tmp:
        llm = LLM()
        async with SelfGraph(store=KuzuStore(f"{tmp}/g.kuzu"), llm=llm) as g:
            await g.insert.text("Ann works at Acme.", source_id="doc", replace_source=True)
            llm.fail = True
            r = await g.insert.text("Ann works at Acme. She likes it.", source_id="doc", replace_source=True)
            print([w.code for w in r.warnings], r.source_diff)
            print([(x.type, x.is_current) for x in (await g.relations.list(include_invalidated=True)).items])

asyncio.run(main())
['extraction_partial_failure', 'fact_retired_by_source'] kept=[] added=[] retired=['relation:a4b698967fceb2c72cc81f83']
[('WORKS_AT', False)]

The TypeScript SDK gives the same warnings and source_diff for the same client.

Cause

Insert.text passes the extraction to _apply_facts with replace_source unchanged, and _ops.replace_source retires prior - produced without knowing that some LLM calls failed. When extraction_partial_failure comes from a failed call (not from dropping one invalid item), replacing the source should either fail or skip retirement for the affected chunks.

主要言語
TypeScript
スター
173
フォーク
58
平均マージ
6時間
マージ済み PR(30日)
120

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

hydra-db/selfgraph のほかの issue

hydra-db/selfgraph の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。