feature: lossless capture in the session archive, so past sessions can be replayed to test a change before turning it on

Start by resolving the open decisions on byte-exact reconstruction, privacy, and retention; then read the proposed PR plan and inspect core/config and core/session/archive for the existing archive format and writer. The plan also names core/listener/forwardproxy, core/pipeline, core/sessionapi, and cmd/agentop as later entry points. Done is not yet a single task: agree on a scoped PR and its tests before starting implementation.

Summary

Cortex sees every request an agent sends to a model, and its session archive already keeps 30 days of them. That makes it the natural place to answer "what would this change have done to my last month of work?" before turning on a plugin, a model, an effort level or a prompt. Today the archive can't answer that question, because it doesn't keep requests in a form that can be sent again.

This issue proposes an opt-in lossless capture mode: the archive keeps the exact request and response bodies, deduplicated so it stays affordable, and a later agentop replay re-sends them with a change applied. It records what is lost today, the measured disk, CPU and memory cost, the decisions to make first, and a 5-PR plan. It is not a design.

Why replay matters: one investigation, measured

A recent investigation into making coding agents faster tested about a dozen ideas by replaying a user's own past turns. Several ideas looked strong in a synthetic benchmark and reversed once replayed on real traffic:

Idea	First result	After replaying real turns
Lower effort on turns that only read tool output	0.11–0.24× task time	0.83× per turn, and each effort switch re-writes the prompt cache
Per-message effort (beta)	Cache-safe	No reduction in thinking at all (1.00×)
Prompting for parallel tool calls	59% of replies batched calls	The extra calls missed the agent's real next step
Lower effort for subagents	0.34× time	A blind judge preferred the full-effort reports 37 of 40 times

None of those replays came from Cortex. Every request was rebuilt from Claude Code's own transcripts. That only works for Claude Code, and it needs guesses for the system prompt and tool definitions, which transcripts don't contain. Cortex has those exact bytes for every agent it proxies, and it has the timings, token counts and failures that only exist on the wire.

What the archive loses today

The inference parser reduces each body to the text it needs for cost and display. Ranked by how badly each loss blocks an exact replay:

1. The raw request bytes. The forward proxy reads the whole body into memory before running plugins, but never stores it. After a plugin rewrites the body (tool-prune, inference-router, context-guru), only the rewritten version remains.
2. Request headers and the query string, such as anthropic-beta, anthropic-version and ?beta=true. SessionEvent has no header field.
3. Tool calls in the conversation history. In a sample of 19,281 stored assistant messages, 65% had no content: they were tool calls, and only their byte count is kept.
4. Thinking blocks and their signatures, and reasoning items (including encrypted_content) in OpenAI's Responses format.
5. The system prompt as separate blocks, and every cache_control marker. The system prompt is flattened into one text message.
6. Top-level parameters: thinking, output_config (effort), stop_sequences, metadata, top_k, service_tier, response_format, previous_response_id, and others.
7. Images, documents, and tool results that aren't text.
8. Byte-exact tool definitions. The schema is re-serialised (compacted and HTML-escaped). Tool type, cache_control and server tools are dropped.
9. The raw response stream. Only the assembled completion and tool calls are kept. Thinking deltas, signatures, citations, message ids and event order are discarded. On the streaming path, each chunk is written to the client and not kept.
10. Requests over the size caps (32 MiB request, 10 MB buffered response), and events dropped when the archive queue is full.

Measured cost

Measured on one laptop's real traffic (Claude Code through a LiteLLM gateway to AWS Bedrock), on 2026-10-22, against main at d4394da7.

Disk

Requests are large because each one re-sends the whole conversation. Storing them whole doesn't work; storing each distinct block once does.

How requests are stored	Per request (614 rebuilt real requests)
Raw body	2,260 KB
Each body compressed on its own (zstd)	1,485 KB
Split into blocks (system, each tool definition, each message block); each distinct block stored once, compressed	21 KB
Raw response stream, compressed (median of 10)	2.3 KB

• The worst session seen had 26 screenshots in its tool results and 4.4 MB requests. With block storage it was still 36 KB per request.
• Claude Code's auto-mode safety check sends the conversation as one large text block. Stored whole, that costs 18.7 KB per check. Stored as what changed since the previous check, it costs 3.8 KB: the median check shares 17 KB of its 25 KB with the one before.

At this user's volume of 1,958 calls a day (5,245 on the busiest day):

	Average day	Busiest day	30 days
Today's archive	~12 MB	—	~0.35 GB
Lossless at 21 KB per request plus the response	46 MB	122 MB	1.4 GB
Lossless at the image-heavy rate (36 KB)	76 MB	203 MB	2.3 GB

That is 4–6× today's archive. It reaches the 2 GiB default session.archive.max_bytes within the 30-day default retention, at which point the oldest sessions are deleted early.

CPU and latency

Go benchmark on a real 639 KB request, on the same laptop:

Step	Time
Split into blocks and hash every block (SHA-256)	2.5 ms, 714 KB allocated
Compress a typical 20 KB of new content (zstd)	0.07 ms
Compress a whole body, for comparison	2.2 ms

That is about 0.02% of the median 10.4 s model call, and about 10 s of CPU a day at this volume. The request doesn't wait for it, as long as the work runs on the archive's background writer and not in the request path.

Memory

• The raw body must not stay in the in-memory session store. Events there stay in RAM, and session.max_events defaults to unlimited. At 2.2 MB per request, a busy day would hold gigabytes.
• The archive queue holds a copy of each event, up to 4,096. If raw bodies ride along, a full queue could pin about 9 GB. The queue needs a byte budget, or the split and hash must happen before the event is queued.
• Copying the response stream needs a buffer per in-flight request, capped at the existing 10 MB frame limit. Responses are small (median 14.5 KB raw).

Decisions to make first

• Which bytes are "the request": what the agent sent, or what Cortex forwarded after plugins changed it? Backtesting a plugin needs the original. Proposal: store both, as the original plus the forwarded body (which shares almost all of its blocks).
• Byte-exact or equivalent? Block storage rebuilds the same JSON, not necessarily the same bytes. Exact bytes matter, because the prompt cache and thinking signatures are sensitive to them. Keeping each request's block order and separators would allow a byte-exact rebuild, verified by a stored hash of the original.
• Privacy:
  • The archive already holds prompts and tool output.
  • Lossless capture adds images, files the agent read, and request headers.
  • Credential headers (authorization, x-api-key, cookie) must be redacted at capture time. Today core/redact applies only to config, not to events.
  • Like the archive itself, this stays local-install only. A cluster needs its own decision about raw bodies on a volume.
• Retention: raise the size cap when lossless mode is on, or give lossless data its own, shorter retention (for example 14 days) so it never pushes out the regular session history.
• Default: off, behind session.archive.lossless.

Proposed plan: 5 PRs, about 3.5–5k lines

#	PR	Main areas	Estimated size
1	Flag and storage format. session.archive.lossless, off by default; a new record kind for blocks plus a per-request manifest (old readers already skip unknown kinds); separate cap and retention	core/config, core/session/archive (format, segment writer, reader), docs	600–900 lines
2	Request capture. Original and forwarded bodies plus redacted headers and query; split and hash on the archive writer; a byte budget on the queue; Anthropic Messages, OpenAI Chat Completions and Responses	core/listener/forwardproxy (body read, request event), core/session/archive, core/pipeline session wire format	900–1,300 lines
3	Response capture. A capped copy of the raw stream on the streaming and buffered paths, compressed and linked to its request; compressed upstream responses handled	core/listener/forwardproxy (streaming and passthrough paths), core/session/archive	500–800 lines
4	Read and export. Rebuild a request from the archive; agentop export --session X --turn N writes a ready-to-send body and verifies its hash; raw bodies kept out of the session API's summaries and live stream	core/session/archive reader, core/sessionapi, cmd/agentop	700–1,000 lines
5	Minimal replay. agentop replay with a built-in noise baseline (the original setting replayed against itself) and cache-busting; per-turn results, no judge yet	cmd/agentop, a new replay package	800–1,200 lines

• Tests are probably 40–50% of each PR.
• PRs 1–3 are the capture work, and useful on their own for debugging. PRs 4–5 turn it into the backtest feature.
• PR 2 is the riskiest, because it touches the request path. It needs a test showing nothing changes when the flag is off, and a benchmark showing no added time before the request goes out.

Lessons a replay command must build in

Each one produced a wrong conclusion during the investigation above until it was handled:

1. A noise baseline. Replay the original setting against itself; otherwise every difference looks real.
2. Gateway response caches. LiteLLM answered identical requests from its cache in 0.2–0.6 s until each replay got a unique stop sequence.
3. Prompt-cache side effects. Changing effort re-wrote the cache once the history contained tool calls. Only a cost readout showed it.
4. Blind judging with swapped order, plus a check that the judge isn't just preferring longer answers.
5. Bad days. One gateway's bad days dominated every latency number, so results from one week may not hold the next.
6. One turn isn't a whole task. Replaying a turn shows the next action, not the path to the result. Full-task replay needs simulated tool results or a real workspace, and is out of scope here.

Not measured

• Disk use for Bob, OpenCode and Codex traffic. These figures are from Claude Code only, rebuilt from transcripts rather than captured by Cortex.
• System prompt and tool-definition sizes for other agents.
• Byte-exact rebuild. The simulation re-serialised JSON, so it shows equivalence, not byte identity.

Related

• #1021: shadowing a request against a second model. Lossless capture lets the same comparison run offline on past traffic, without a live second request.
• #1417: counting savings from request-rewriting plugins. Storing the original and the forwarded body lets a replay measure those savings directly.
• #1473: re-sending stalled requests. Its evidence came from the archive's timings.

Assisted-By: Claude (Anthropic AI) [email protected]