agents-runtime: openai reasoning items dropped across wakes — LLMMessage schema gap

Open
#4,390 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
62/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
typescript
Domain
api, backend

Research direction

Start by reading packages/agents-runtime/src/types.ts, pi-adapter.ts, and outbound-bridge.ts, then trace the existing function-call and tool-result round-trip. Reproduce the multi-wake reasoning-model scenario described in the issue. Done means the reasoning signature is preserved and rebuilt with assistant history, so a follow-up wake with a paired function call no longer returns the missing rs_ item error.

Written by the indexing model from the issue text.

Description

Summary

When a long-lived agents-runtime entity uses an OpenAI reasoning model (e.g. gpt-5.5, o-series) on a multi-wake conversation, the second wake fails with:

```
404 Item with id 'rs_' not found.
Items are not persisted when `store` is set to false.
```

The underlying LLM library (`@mariozechner/pi-ai`) handles reasoning correctly. The break is a schema gap in `@electric-ax/agents-runtime` that drops the encrypted reasoning blob between wakes — but keeps the paired `function_call` id, so OpenAI gets an unmatched `fc_` and rejects it.

Root cause

  1. pi-ai sends `include: ["reasoning.encrypted_content"]` for reasoning providers (`pi-ai/dist/providers/openai-responses.js:178`). The Responses API returns each `rs_` item with its full encrypted payload.

  2. pi-ai stashes that blob as a JSON string in `block.thinkingSignature` (`pi-ai/dist/providers/openai-responses-shared.js:367-369`) and, on the next request, `JSON.parse()`s it back into the `input` array (`openai-responses-shared.js:109-114`). This is the contract for stateless reasoning — no `store: true` required.

  3. agents-runtime's `LLMMessage` schema has nowhere to persist `thinkingSignature`. In `packages/agents-runtime/src/types.ts:216-245`:

    • `LLMAssistantMessage` carries `content: string`. No `thinkingSignature`, no `reasoning` variant, no `encrypted_content` field.
    • `pi-adapter.ts:86-160` `toAgentHistory()` rebuilds `AgentMessage[]` from `LLMMessage[]` on every wake. The assistant case (lines 107-122) only emits `{type: 'text'}` blocks. The pi-ai reasoning round-trip is broken at exactly the persistence boundary.
  4. But the paired `function_call` id (`fc_`) IS preserved (`pi-adapter.ts:124-144`). OpenAI's Responses API tracks `fc_`↔`rs_` pairs and rejects an `fc_` submission without its companion `rs_`. So on the second wake, pi-adapter sends the `function_call` id without the reasoning item, and the API 404s on the missing `rs_`.

Why the bundled paths don't hit this

The built-in catalog (`packages/agents/src/model-catalog.ts`) avoids the path by default:

  • Anthropic is preferred when `ANTHROPIC_API_KEY` is set (`model-catalog.ts:209-222`). Anthropic returns no `rs_` items.
  • The OpenAI fallback is `DEFAULT_OPENAI_MODEL = 'gpt-4.1'` (`model-catalog.ts:49`), which has `reasoning: false` in the generated catalog → `withProviderPayloadDefaults` short-circuits at `model-catalog.ts:148-152` and no `reasoning.effort` is sent → no `rs_` items.
  • `openai-codex` paths run single-turn or rely on the codex token's own store semantics.

The gap surfaces when:

  • a host application explicitly picks an OpenAI reasoning model (e.g. `gpt-5.5`) for an entity that
  • runs a multi-wake conversation with tool calls (so a paired `fc_` lands in history).

Repro

  1. Register an entity-type with a model catalog whose default choice is `{ provider: 'openai', id: 'gpt-5.5', reasoning: true }`.
  2. Spawn one instance. Have it issue a tool call on its first wake (so the wake history records an `fc_` paired with an `rs_`).
  3. Send a follow-up message. On the second wake, `pi-adapter.toAgentHistory` rebuilds messages without the `thinking` block; the model call goes out carrying `fc_` but no `rs_`. Responses API returns `404 Item with id 'rs_' not found`.

Proposed fix

Three coordinated changes in `packages/agents-runtime/src/`:

  1. `types.ts:230` — add an optional `thinkingSignature?: string` to `LLMAssistantMessage`, or model a separate `thinking` content-variant alongside `text` and `tool_use`. Either shape will round-trip pi-ai's blob.
  2. `outbound-bridge.ts` — capture pi-agent-core's `thinking_end` events (which carry `contentSignature`) and write them into the durable event stream alongside the assistant text.
  3. `pi-adapter.ts:107-122` — in `toAgentHistory`, when an assistant message carries a `thinkingSignature`, prepend a `{type: 'thinking', thinking: '', thinkingSignature}` block before the text block so pi-ai sees it on rebuild.

After that, `gpt-5.5` (and other reasoning OpenAI models) should run multi-wake without errors, matching what the existing fc_/tool-result round-trip already does.

Workaround for now

Set `reasoning: false` on the OpenAI catalog choice. The runtime then never asks for reasoning content, no `rs_` items are emitted, and the gap is invisible. This matches what the built-in `gpt-4.1` default already does — it's the only validated openai path today.

Versions

  • `@electric-ax/agents-runtime` 0.3.0
  • `@electric-ax/agents` 0.4.4
  • `@mariozechner/pi-ai` 0.70.2 — 0.70.6 (relevant dist files are byte-identical between these versions)
  • `@mariozechner/pi-agent-core` 0.70.2 — 0.70.6
Dominant language
TypeScript
Stars
10.4k
Forks
399
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from electric-sql/electric

All issues in electric-sql/electric

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.