boundImages recomputes the strip set per request — stripped images flip back to live and invalidate the prompt cache

Summary

boundImages() (packages/core/src/session/model-request.ts, since v2.0.x) recomputes which inline images to strip from scratch on every request. When the history changes (a new image arrives, compaction prunes, tool output is truncated), the total image bytes move around the 25 MiB trigger and previously-stripped images flip back to live. That rewrites the prompt prefix at the oldest image's position, so every provider-side prompt cache behind it is discarded — in our case a ~300k-token conversation re-prefilled on every turn after a few screenshots were read, including plain-text turns.

Evidence

With a local inference server that logs cache reuse, the divergence is exactly at the position of the first stripped image: consecutive requests alternate between the image payload and the IMAGE_REMOVED placeholder at the same token offset (image_pad vs placeholder text), and cache reuse collapses from ~99.9% to the system-prompt root each time. On our backend (strata, a custom llama.cpp-derived server with a conversation KV cache) this forces a full re-prefill of the entire KV cache — every turn after a few screenshots, including plain-text turns, pays ~300k tokens of prefill (~30–40 s) that the cache should have served.

Why it's a bug, not a policy question

The strip policy (25 MiB trigger → 15 MiB target) is untouched. The issue is that the decision is non-monotonic: the same message renders differently across requests, which silently breaks prefix caching for every provider. A one-way decision fixes it without changing when stripping happens.

Proposed fix

Make the strip decision sticky per image (content hash + occurrence index, so identical-but-distinct attachments stay distinct): once an image is replaced by the placeholder, it stays replaced for the process lifetime. Stripping still only starts above the trigger and winds down to the target.

commit 76a65c3b009c71d5d88e0819414ab8f78b787666
Author: hermes <hermes@local>
Date:   Mon Oct 5 15:09:49 2026 +0800

    fix(session): make boundImages stripping sticky per image
    
    Recomputing the strip set per request let a stripped image flip back to
    live whenever the history changed, rewriting the prompt prefix at the
    image position and discarding the provider prompt cache behind it. The
    decision is now one-way, keyed by content hash + occurrence.

diff --git a/packages/core/src/session/model-request.ts b/packages/core/src/session/model-request.ts
index 26cecde..e5c8542 100644
--- a/packages/core/src/session/model-request.ts
+++ b/packages/core/src/session/model-request.ts
@@ -158,6 +158,32 @@ export const unsupportedParts = (messages: LLMRequest["messages"], capabilities:
     }),
   )
 
+// Images already replaced by the placeholder in this process. Recomputing the strip set per request
+// let a stripped image flip back to live whenever the history changed (a new image, compaction, or
+// pruning moved the total back under the trigger), rewriting the prompt prefix at the image's position
+// and discarding the provider's prompt cache for the whole conversation behind it. The decision is
+// one-way: once stripped, an image stays stripped for the life of the process.
+const strippedImages = new Set<string>()
+
+// A stable identity for an image payload: mime, length, and sampled bytes. Identical bytes across
+// requests hash identically, so the sticky decision survives every rebuild of the request.
+const imageKey = (mime: string, sample: string, length: number) => {
+  let h = 0xcbf29ce484222325n
+  const s = `${mime}:${length}:${sample}`
+  for (let i = 0; i < s.length; i++) {
+    h ^= BigInt(s.charCodeAt(i))
+    h = (h * 0x100000001b3n) & 0xffffffffffffffffn
+  }
+  return h.toString(16)
+}
+
+const mediaSample = (media: Media.Asset) => {
+  if (media.source.type === "base64") return imageKey(media.mediaType, media.source.data.slice(0, 4096), media.source.data.length)
+  if (media.source.type === "bytes")
+    return imageKey(media.mediaType, media.source.data.subarray(0, 4096).toString("base64"), media.source.data.byteLength)
+  return imageKey(media.mediaType, media.source.type, 0)
+}
+
 export const boundImages = (messages: LLMRequest["messages"]) => {
   const isImage = (mime: string) => mime.toLowerCase().startsWith("image/")
   // Remote and provider-referenced media carry no local payload and never count toward the inline budget.
@@ -166,33 +192,58 @@ export const boundImages = (messages: LLMRequest["messages"]) => {
     if (media.source.type === "bytes") return Math.ceil(media.source.data.byteLength / 3) * 4
     return 0
   }
-  const imageBytes = messages.reduce(
-    (total, message) =>
-      total +
-      message.content.reduce((sum, part) => {
-        if (part.type === "media" && isImage(part.media.mediaType)) return sum + size(part.media)
-        if (part.type !== "tool-result" || part.result.type !== "content") return sum
-        return (
-          sum +
-          part.result.value.reduce(
-            (bytes: number, item: Content) =>
-              bytes + (item.type === "file" && isImage(item.mime) ? Buffer.byteLength(item.uri) : 0),
-            0,
-          )
-        )
-      }, 0),
-    0,
-  )
-  if (imageBytes <= IMAGE_BYTES_TRIGGER) return messages
+  const uriSize = (uri: string) => Buffer.byteLength(uri)
+
+  // Identical payloads are distinct attachments, so each occurrence of a hash gets its own key.
+  const occurrence = () => {
+    const seen = new Map<string, number>()
+    return (base: string) => {
+      const n = seen.get(base) ?? 0
+      seen.set(base, n + 1)
+      return `${base}#${n}`
+    }
+  }
+
+  // Pass 1: bytes of the images still live - sticky-stripped ones no longer count.
+  let live = 0
+  const counted = occurrence()
+  for (const message of messages)
+    for (const part of message.content) {
+      if (part.type === "media" && isImage(part.media.mediaType)) {
+        if (!strippedImages.has(counted(mediaSample(part.media)))) live += size(part.media)
+      } else if (part.type === "tool-result" && part.result.type === "content") {
+        for (const item of part.result.value) {
+          if (item.type === "file" && isImage(item.mime) && !strippedImages.has(counted(imageKey(item.mime, item.uri.slice(0, 4096), item.uri.length))))
+            live += uriSize(item.uri)
+        }
+      }
+    }
 
-  let removed = 0
-  return messages.map((message) =>
+  // Pass 2: rebuild. Sticky images stay placeholders; if still over the trigger, strip oldest-first
+  // down to the target and remember the decision.
+  if (live <= IMAGE_BYTES_TRIGGER && strippedImages.size === 0) return messages
+  let budget = live
+  let changed = false
+  const overTrigger = live > IMAGE_BYTES_TRIGGER
+  const shouldStrip = () => overTrigger && budget > IMAGE_BYTES_TARGET
+  const placed = occurrence()
+  const out = messages.map((message) =>
     Message.make({
       ...message,
       content: message.content.map((part) => {
-        if (part.type === "media" && isImage(part.media.mediaType) && imageBytes - removed > IMAGE_BYTES_TARGET) {
-          removed += size(part.media)
-          return Message.text(IMAGE_REMOVED)
+        if (part.type === "media" && isImage(part.media.mediaType)) {
+          const key = placed(mediaSample(part.media))
+          if (strippedImages.has(key)) {
+            changed = true
+            return Message.text(IMAGE_REMOVED)
+          }
+          if (shouldStrip()) {
+            strippedImages.add(key)
+            budget -= size(part.media)
+            changed = true
+            return Message.text(IMAGE_REMOVED)
+          }
+          return part
         }
         if (part.type !== "tool-result" || part.result.type !== "content") return part
         return {
@@ -200,15 +251,27 @@ export const boundImages = (messages: LLMRequest["messages"]) => {
           result: {
             ...part.result,
             value: part.result.value.map((item: Content) => {
-              if (item.type !== "file" || !isImage(item.mime) || imageBytes - removed <= IMAGE_BYTES_TARGET) return item
-              removed += Buffer.byteLength(item.uri)
-              return { type: "text" as const, text: IMAGE_REMOVED }
+              if (item.type !== "file" || !isImage(item.mime)) return item
+              const key = placed(imageKey(item.mime, item.uri.slice(0, 4096), item.uri.length))
+              if (strippedImages.has(key)) {
+                changed = true
+                return { type: "text" as const, text: IMAGE_REMOVED }
+              }
+              if (shouldStrip()) {
+                strippedImages.add(key)
+                budget -= uriSize(item.uri)
+                changed = true
+                return { type: "text" as const, text: IMAGE_REMOVED }
+              }
+              return item
             }),
           },
         }
       }),
     }),
   )
+  // Nothing was replaced: hand back the original array, as the untouched fast path does.
+  return changed ? out : messages
 }
 
 type Definitions = PluginHooks.Domains["session"]["context"]["tools"]
diff --git a/packages/core/test/session-model-request.test.ts b/packages/core/test/session-model-request.test.ts
index fb3570e..5043e07 100644
--- a/packages/core/test/session-model-request.test.ts
+++ b/packages/core/test/session-model-request.test.ts
@@ -85,6 +85,20 @@ describe("SessionModelRequest.boundImages", () => {
     expect(result[2]?.content[0]).toMatchObject({ type: "media", filename: "third.png" })
   })
 
+  test("keeps stripped images stripped when the history shrinks back under the trigger", () => {
+    const image = "b".repeat(9 * 1024 * 1024)
+    const mk = (name: string) => Message.user({ type: "media", media: Media.base64(image, "image/png"), filename: name })
+    const first = boundImages([mk("a"), mk("b"), mk("c")])
+    expect(first[0]?.content[0]).toMatchObject({ type: "text" })
+    expect(first[1]?.content[0]).toMatchObject({ type: "text" })
+    expect(first[2]?.content[0]).toMatchObject({ type: "media", filename: "c" })
+    // Same history replayed: the stripped images must not flip back to live and rewrite the prefix.
+    const again = boundImages([mk("a"), mk("b"), mk("c")])
+    expect(again[0]?.content[0]).toMatchObject({ type: "text" })
+    expect(again[1]?.content[0]).toMatchObject({ type: "text" })
+    expect(again[2]?.content[0]).toMatchObject({ type: "media", filename: "c" })
+  })
+
   test("replaces images nested in tool results", () => {
     const image = "a".repeat(13 * 1024 * 1024)
     const result = boundImages([

Also adds a regression test: replay a stripped history and assert the stripped images do not come back. All existing boundImages tests pass unchanged.

Happy to open a PR if this shape is acceptable — and thanks for the great work on 2.x 🙏