agents-runtime: openai reasoning items dropped across wakes — LLMMessage schema gap

Aperta
#4,390 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
62/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Tranquilla
Stack tecnologico
typescript
Ambito
api, backend

Direzione di ricerca

Inizia leggendo packages/agents-runtime/src/types.ts, pi-adapter.ts e outbound-bridge.ts, quindi segui il round-trip esistente della chiamata di funzione e del risultato dello strumento. Riproduci lo scenario con un modello di reasoning e più wake descritto nell’issue. Il lavoro è completato quando la firma di reasoning viene preservata e ricostruita insieme alla cronologia dell’assistant, in modo che un wake successivo con una chiamata di funzione associata non restituisca più l’errore missing rs_ item.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

When a long-lived agents-runtime entity uses an OpenAI reasoning model (e.g. gpt-5.5, o-series) on a multi-wake conversation, the second wake fails with:

```
404 Item with id 'rs_' not found.
Items are not persisted when `store` is set to false.
```

The underlying LLM library (`@mariozechner/pi-ai`) handles reasoning correctly. The break is a schema gap in `@electric-ax/agents-runtime` that drops the encrypted reasoning blob between wakes — but keeps the paired `function_call` id, so OpenAI gets an unmatched `fc_` and rejects it.

Root cause

  1. pi-ai sends `include: ["reasoning.encrypted_content"]` for reasoning providers (`pi-ai/dist/providers/openai-responses.js:178`). The Responses API returns each `rs_` item with its full encrypted payload.

  2. pi-ai stashes that blob as a JSON string in `block.thinkingSignature` (`pi-ai/dist/providers/openai-responses-shared.js:367-369`) and, on the next request, `JSON.parse()`s it back into the `input` array (`openai-responses-shared.js:109-114`). This is the contract for stateless reasoning — no `store: true` required.

  3. agents-runtime's `LLMMessage` schema has nowhere to persist `thinkingSignature`. In `packages/agents-runtime/src/types.ts:216-245`:

    • `LLMAssistantMessage` carries `content: string`. No `thinkingSignature`, no `reasoning` variant, no `encrypted_content` field.
    • `pi-adapter.ts:86-160` `toAgentHistory()` rebuilds `AgentMessage[]` from `LLMMessage[]` on every wake. The assistant case (lines 107-122) only emits `{type: 'text'}` blocks. The pi-ai reasoning round-trip is broken at exactly the persistence boundary.
  4. But the paired `function_call` id (`fc_`) IS preserved (`pi-adapter.ts:124-144`). OpenAI's Responses API tracks `fc_`↔`rs_` pairs and rejects an `fc_` submission without its companion `rs_`. So on the second wake, pi-adapter sends the `function_call` id without the reasoning item, and the API 404s on the missing `rs_`.

Why the bundled paths don't hit this

The built-in catalog (`packages/agents/src/model-catalog.ts`) avoids the path by default:

  • Anthropic is preferred when `ANTHROPIC_API_KEY` is set (`model-catalog.ts:209-222`). Anthropic returns no `rs_` items.
  • The OpenAI fallback is `DEFAULT_OPENAI_MODEL = 'gpt-4.1'` (`model-catalog.ts:49`), which has `reasoning: false` in the generated catalog → `withProviderPayloadDefaults` short-circuits at `model-catalog.ts:148-152` and no `reasoning.effort` is sent → no `rs_` items.
  • `openai-codex` paths run single-turn or rely on the codex token's own store semantics.

The gap surfaces when:

  • a host application explicitly picks an OpenAI reasoning model (e.g. `gpt-5.5`) for an entity that
  • runs a multi-wake conversation with tool calls (so a paired `fc_` lands in history).

Repro

  1. Register an entity-type with a model catalog whose default choice is `{ provider: 'openai', id: 'gpt-5.5', reasoning: true }`.
  2. Spawn one instance. Have it issue a tool call on its first wake (so the wake history records an `fc_` paired with an `rs_`).
  3. Send a follow-up message. On the second wake, `pi-adapter.toAgentHistory` rebuilds messages without the `thinking` block; the model call goes out carrying `fc_` but no `rs_`. Responses API returns `404 Item with id 'rs_' not found`.

Proposed fix

Three coordinated changes in `packages/agents-runtime/src/`:

  1. `types.ts:230` — add an optional `thinkingSignature?: string` to `LLMAssistantMessage`, or model a separate `thinking` content-variant alongside `text` and `tool_use`. Either shape will round-trip pi-ai's blob.
  2. `outbound-bridge.ts` — capture pi-agent-core's `thinking_end` events (which carry `contentSignature`) and write them into the durable event stream alongside the assistant text.
  3. `pi-adapter.ts:107-122` — in `toAgentHistory`, when an assistant message carries a `thinkingSignature`, prepend a `{type: 'thinking', thinking: '', thinkingSignature}` block before the text block so pi-ai sees it on rebuild.

After that, `gpt-5.5` (and other reasoning OpenAI models) should run multi-wake without errors, matching what the existing fc_/tool-result round-trip already does.

Workaround for now

Set `reasoning: false` on the OpenAI catalog choice. The runtime then never asks for reasoning content, no `rs_` items are emitted, and the gap is invisible. This matches what the built-in `gpt-4.1` default already does — it's the only validated openai path today.

Versions

  • `@electric-ax/agents-runtime` 0.3.0
  • `@electric-ax/agents` 0.4.4
  • `@mariozechner/pi-ai` 0.70.2 — 0.70.6 (relevant dist files are byte-identical between these versions)
  • `@mariozechner/pi-agent-core` 0.70.2 — 0.70.6
Lingua principale
TypeScript
Stelle
10.4k
Fork
399
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di electric-sql/electric

Tutte le issue di electric-sql/electric

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.