TypeScript extraction rejects LLM replies that Python accepts

Aperta
#361 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
68/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, typescript
Ambito
api, backend

Direzione di ricerca

Read typescript/src/extraction/extractor.ts alongside src/selfgraph/extraction/extractor.py, focusing on the output models named in the issue and the strength validators. Compare the defaults and accepted values across the four extraction passes, then run the reproduction against embedded Kuzu. Done when the TypeScript insert accepts the omitted fields and strength cases described and behaves consistently with Python.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

approved-for-pr area: extraction area: parity bug effort: S P2

What happens

The TypeScript extractor's output schemas mark optional fields as .nullable(), so the key must be present. Python's models give the same fields defaults. A reply that leaves out aliases, event_time, about_name, strength, actor_name, summary, rationale, outcome_name, alternative_names, status, or steps is accepted in Python and fails the whole insert.text in TypeScript with invalid_llm_output. stance defaults to preference in Python and is required in TypeScript.

Number handling for strength differs too. Python's _PrefOut.strength is a lax float: true and false are stored as 1.0 and 0.0, although insert.facts rejects a boolean strength (StrictFloat). NaN and infinity reach _validate_strength and are dropped per item. TypeScript rejects all of these at the output schema, failing the insert.

Providers in JSON or tool mode often omit null fields, so the same client works with one SDK and not the other.

Reproduce

On main (5a277c9), embedded Kuzu:

import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { KuzuStore, SelfGraph } from "@hydradb/selfgraph";

async function run(name: string, replies: Record<string, unknown[]>) {
  const llm = {
    async generate({ output }: { output: { shape: Record<string, unknown> } }) {
      const kind = Object.keys(output.shape)[0]!;
      return { [kind]: replies[kind] ?? [] };
    },
  };
  const g = new SelfGraph(new KuzuStore(join(mkdtempSync(join(tmpdir(), "sg-")), "g.kuzu")), { llm });
  try {
    const r = await g.insert.text("Ann works at Acme.");
    console.log(name, "ok", r.entities.map((e) => e.id));
  } catch (e) {
    console.log(name, (e as { code?: string }).code, (e as Error).message);
  } finally {
    await g.close();
  }
}

await run("missing aliases", {
  entities: [{ type: "Person", name: "Ann" }, { type: "Company", name: "Acme" }],
  relations: [{ from_name: "Ann", type: "WORKS_AT", to_name: "Acme", event_time: null }],
});
await run("missing stance", {
  preferences: [{ subject_name: "Ann", key: "food", value_name: "Pizza", about_name: null, strength: null }],
});
await run("boolean strength", {
  preferences: [{ subject_name: "Ann", stance: "preference", key: "food", value_name: "Pizza", about_name: null, strength: true }],
});
missing aliases invalid_llm_output LLM client returned an invalid result for the entities pass: entities[0].aliases: field required
missing stance invalid_llm_output LLM client returned an invalid result for the preferences pass: preferences[0].stance: expected 'preference' or 'disinterest'
boolean strength invalid_llm_output LLM client returned an invalid result for the preferences pass: preferences[0].strength: expected number

Python inserts all three: person:ann and company:acme with the relation, a preference with stance preference, and a preference with strength 1.0.

Cause

EntitiesOut, RelationsOut, PreferencesOut, and DecisionsOut in typescript/src/extraction/extractor.ts versus _EntityOut, _RelOut, _PrefOut, and _DecOut in src/selfgraph/extraction/extractor.py. The two need the same optional fields and defaults, and strength the same strictness in both.

Lingua principale
TypeScript
Stelle
173
Fork
58
Merge medio
6h
PR unite (30g)
120

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di hydra-db/selfgraph

Tutte le issue di hydra-db/selfgraph

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.