Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

bug(pi): compaction recovery notice in the system prompt forces a full prompt re-read on the next two user turns

Open
  • Is something wrong?
#1,894 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
Half a day
Newbie friendliness
58/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
typescript
Domain
ai-infra-agents

Research direction

The relevant code is in plugin/pi/index.ts: the session_compact handler that sets pendingRecoveryNotice (around lines 2937 and 2962) and the before_agent_start handler that appends it to the system prompt (lines 2988-2991). Start by reading how appendPromptSection and the returned result are built in the before_agent_start handler. Done means the recovery notice reaches the model without changing the system prompt across the next user turns, which you can check by confirming the prompt prefix stays identical between requests.

Written by the indexing model from the issue text.

Description

status:needs-review type:bug
Existing Issue Search
  • I searched existing issues and will add evidence to a canonical tracker when applicable.
📝 Observed Consequence

With the Pi plugin and a local OpenAI-compatible server that caches prompt prefixes, the first two user replies after every compaction re-read the whole conversation instead of reusing the cache. In one session each of those turns re-read ~118k-119k tokens (~106-108 s each), while user replies without that change reused ~99% of the prompt in 5-8 s.

Both turns change only one part of the system prompt: the gentle-engram compaction recovery notice (## Memory from This Session ... / Compaction recovery summary was already saved to Engram ...). On the first reply the previous notice is replaced by a new one; on the second reply it is removed.

🔄 Steps to Reproduce
  1. Use Pi with gentle-engram and a model served by a local server with prompt (prefix) caching, in a long session.
  2. Let an auto-compaction happen during an autonomous run (no user message in between).
  3. Send a user message, then another one. Compare each request's cached/reused prompt tokens on the server with the previous request.
✅ Expected Behavior

The recovery notice reaches the model without changing the system prompt, so the cached prefix stays valid across user turns.

❌ Actual Behavior

Both user turns after a compaction get no cache reuse; the server re-reads the full prompt (table below, from the server log, request sizes matched 1:1 with the session's input + cacheRead tokens).

Turn System prompt change (replayed from pi's persisted role: "system" patch entries) Request Server prompt cache
Last tool turn of the autonomous run — 115597 tokens 112919 reused
1st user reply text after MEMORY_INSTRUCTIONS: old notice (1234 chars) -> new notice (1316 chars); first changed char 51168 of 63228 117764 tokens 0 reused + 117764 read in 105992 ms
2nd user reply new notice -> nothing (0 chars); first changed char 51005 of 61910 119392 tokens 0 reused + 119392 read in 107734 ms
Next tool turn unchanged 121030 tokens 118049 reused

Control, same session: user replies that did not change the system prompt reused 127888/128568, 177734/180214, 185512/187704 and 189219/192405 tokens. Tools did not change on the affected turns.

Operating System

Windows

Release Version (Report Context)

gentle-engram 0.3.0 (Pi plugin, npm). The same code is on current main 82846a101296e75351ae444953438cace88e9bea (plugin/pi 0.4.2), plugin/pi/index.ts lines 2937, 2962 and 2988-2991.

Agent / Client

Other

📋 Relevant Logs
# server log (prompt cache), consecutive requests around the two user replies
prompt 115597 tokens = 112919 reused + 2678 read
prompt 117764 tokens = 0 reused + 117764 read in 105992 ms
prompt 119392 tokens = 0 reused + 119392 read in 107734 ms
prompt 121030 tokens = 118049 reused + 2981 read in 7571 ms
💡 Additional Context

Client: Pi coding agent (via gentle-shell) with a local llama.cpp-style server.

Where it comes from (on main 82846a1):

  • session_compact handler sets pendingRecoveryNotice = { sessionId, content: buildRecoveryNotice(...) } (plugin/pi/index.ts 2937, 2962).
  • before_agent_start handler: appendPromptSection(options, pendingRecoveryNotice.content) (or appends it to the returned systemPrompt when systemPromptOptions is absent), then clears it (2988-2991).
  • pi renders appendSystemPrompt in the addendum section, followed by project_context, skills, cwd and mcp_servers, so the change sits ~81% into the system prompt, before every conversation message. The system prompt is not rebuilt during an autonomous run, so the notice stays for the whole run and changes again on the next user replies. On this server any change means a full re-read; a longest-common-prefix cache would still re-read the whole conversation.

Related: #1588 / #1533 moved the protocol and the notice into systemPromptOptions.appendSystemPrompt so triggerTurn turns keep the protocol. The suggestion below keeps that for MEMORY_INSTRUCTIONS and only changes how the one-shot notice is delivered.

Possible fix: deliver the notice as a turn message. pi supports BeforeAgentStartEventResult.message (Pick<CustomMessage, "customType" | "content" | "display" | "details">), and convertToLlm sends custom messages to the model as a user message. The handler returns result from several early exits, so the message should be part of result itself:

const notice = pendingRecoveryNotice !== undefined && sessionId === pendingRecoveryNotice.sessionId
  ? pendingRecoveryNotice.content
  : undefined;
if (notice !== undefined) pendingRecoveryNotice = undefined;
const message = notice !== undefined
  ? { customType: "engram.compaction-recovery", content: notice, display: false }
  : undefined;
// The notice is no longer appended to the system prompt in either branch.
const result = options
  ? (message ? { message } : undefined)
  : { systemPrompt, ...(message ? { message } : {}) };

Workaround in use: a local Pi extension loaded after gentle-engram that finds where the previous turn's appendSystemPrompt ended inside the new one and moves anything appended after that point into a hidden turn message. Replayed on the whole session, both user turns keep the system prompt unchanged. It is a heuristic; the fix above removes the need for it.

Dominant language
Go
Stars
7.1k
Forks
729
Avg merge
11h 17m
Merged PRs (30d)
334

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Gentleman-Programming/engram

All issues in Gentleman-Programming/engram

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.