Compile: one root relations.jsonl caps knowledge-graph size (collisions must rewrite the whole file within max_tokens)

Aperta
  • Qualcosa non va?
#7,300 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
json, python
Ambito
backend, data

Direzione di ricerca

Read bot/vikingbot/compile/ops/reduce.py around resolve_files and reduce_group, then bot/vikingbot/compile/pipeline_io.py for the output limit. Compare the collision merge and incremental refresh paths with the knowledge-graph Skill’s relations.jsonl output. Done means large graphs can be merged or refreshed beyond one response’s token limit; the issue lists deterministic merging, append or patch outputs, and sharding as possible approaches.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Version
  • OpenViking core 0.5.3 (178a8cdfb)
  • Compile: bot/vikingbot/compile from main 4b92b3065
  • LiteLLM 1.102.4, OpenAI model, vlm.max_tokens left at the default
  • Workload: examples/compile/ov-compile-skills/knowledge-graph Skill, directory compile, 25 to 45 transcript files per batch
Problem

The knowledge-graph Skill writes every edge to a single root relations.jsonl (SKILL.md, "Typed edges", and the final check "the final artifact tree contains the complete entity set and relations.jsonl"). Every Reduce group that produces edges therefore emits a candidate at the same path, and every incremental refresh recalls the existing file.

Both paths require the model to write the whole file in one response:

  1. Same-path collisions: resolve_files in bot/vikingbot/compile/ops/reduce.py (around L98 to L160) asks the model to "Submit complete files ... without patches or base_hash". The merged relations.jsonl must fit in one model output.
  2. Incremental refresh: reduce_group (around L218 to L226) loads each recalled target file whole into old[path] with no size budget, and the model must again return the complete file.
  3. The output ceiling is vlm.max_tokens, DEFAULT_MAX_TOKENS = 32_000 in bot/vikingbot/compile/pipeline_io.py (L37). A response that stops with finish_reason length raises "Output limit reached; reduce result size or use file references for agent files" (around L500).

At roughly 250 bytes per edge line, 32k output tokens allows about 400 to 500 edges before a collision or refresh of relations.jsonl can no longer be written. Our corpus projects to 3,400 to 10,000 edges, so a single target cannot hold it. Entity pages do not hit this because they are one file per entity.

Minimal reproduction

We avoided this path with the workaround below, so the failure here is derived from the code path and the size projection rather than a captured run log.

  1. Run ov compile with the knowledge-graph Skill on a directory large enough to produce more than one Reduce group, with combined edges above about 500.
  2. Each group emits relations.jsonl; resolve_files sends all candidates to one reduce_merge call.
  3. Expected from the code: the merge response hits the output limit (finish_reason=length), pipeline_io.py raises "Output limit reached", and resolve_files records Unresolved output path relations.jsonl and marks the records behind it failed. Retrying (as in PR #7124) cannot help, since the required output is larger than the limit.
  4. The same applies to an incremental refresh of a target whose existing relations.jsonl is already near the limit.
Expected

Graph size should be bounded by storage and input budgets, not by one model response.

Workaround we use

We compile each batch into its own target, then merge the per-batch graphs outside Compile with a deterministic script (no model): entity pages are joined by normalized id, edges are keyed by (from, relation, to) with their evidence lists united, and the result is checked with examples/compile/graph-show/knowledge-graph/knowledge_graph.py before publishing. This works, but it duplicates logic Compile already has and gives up incremental refresh.

Ask

Any one of these would remove the ceiling:

  • Merge line-oriented files such as relations.jsonl deterministically in resolve_files (union of lines keyed by from, relation, to, evidence united), instead of a model rewrite.
  • Allow append or patch outputs (for example base_hash plus added lines) for collision and refresh outputs, so the model only writes new edges.
  • Or change the Skill to shard edges, for example one edge file per source entity (relations/<from-id>.jsonl), so each file stays small.

Related: #6370 (Reduce input injection has no budget), #7124 (recovery after output exhaustion).

Lingua principale
Python
Stelle
41.8k
Fork
3.4k
Merge medio
19h 29m
PR unite (30g)
500

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di volcengine/OpenViking

Tutte le issue di volcengine/OpenViking

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.