conversation cache: once every slot holds a pinned prefix, nothing is parked any more ("skip parking ... exceeds available budget")

include/strata/core/conversation_cache.hpp の evict_oldest() とその呼び出し元の make_room() から始め、次に issue が引用している src/core/conversation_cache_test.cpp のテストブロックを読んでください。提案された変更を適用し、ピン保持者が別に残っている間もピン留めされた対象が退避されるようにし、make_room() のコメントを更新してください。新しいチェック "fourth pinned one still parks" が通り、既存のピン留めプレフィックスのチェックも引き続き通れば完了です。

Version: 0.1.42 (single RTX 3090, Linux, CUDA). evict_oldest() is unchanged on main (a0116b27, 2026-10-12).

Setup: Home Assistant's voice assistant talks to Strata through an OpenAI-compatible gateway that adds strata_prefix to every request, so the system prompt + tool list (~6–10K tokens) stay pinned. --conversation-cache-mib 22528 --conversation-cache-slots 32, "parallel": 2. Other clients (agents, long-context jobs) share the engine.

What happens: every Home Assistant conversation carries the pinned checkpoint in its chain, so SavedConversation::pinned() is true for all of them. make_room() → evict_oldest() only evicts entries that are not pinned. Once all 32 slots held pinned conversations, evict_oldest() returned false and every attempt to park was refused:

strata serve: conversation cache: skip parking (snapshot 451 MiB exceeds available budget)

(315 times in the ~2,000 log lines after the last successful park, no park in between, snapshots from 225 to 3,806 MiB alike, while the parked bytes were below the 22 GiB budget and ~11 GiB of RAM were still available — so the slot count with only pinned entries, not the byte budget, refused them). The last successful park before that read parked=32 ... evictions=309. Effects: a conversation only resumed from the cache while it was still the active one; everything else was read again — ~4 s instead of ~0.4–0.8 s for a 7K voice request, and three long-context requests (186K–202K tokens) were read from token 0 (~90–98 s each).

Relation to #1559: #1559 retires stale pins when a new prefix replaces the old one. In our log the pin position changed several times (4578, 5751, 5970, 7250, 6484 tokens — the Home Assistant prompt changes when entities are renamed or exposed), so stale pins probably contributed, and #1559 may already fix part of what we saw; I have not run with #1559 applied. What #1559 cannot cover: a client with a stable prefix. Every parked conversation then holds the current pin, #1559 keeps them all pinned, and once slots conversations are parked evict_oldest() still returns false. This patch is for that case and is independent of #1559.

Proposed fix (minimal): when only pinned entries are left, evict the oldest pinned one as long as another holder of a pin remains (the newest is kept), so the shared prefix stays parked and findable. Note that make_room() calls evict_oldest() in a loop, so a very large incoming snapshot can clear all pinned entries but the newest one, not just one.

--- a/include/strata/core/conversation_cache.hpp
+++ b/include/strata/core/conversation_cache.hpp
@@
-    // False when none can go (empty, or only entries that hold a pinned shared prefix are left).
+    // False when none can go (empty, or a single entry that holds a pinned shared prefix is left).
     bool evict_oldest() {
         auto victim = std::find_if(entries_.begin(), entries_.end(), [](const SavedConversation& e) { return !e.pinned(); });
-        if (victim == entries_.end()) return false;
+        if (victim == entries_.end()) {
+            // Every parked conversation holds a pinned prefix (a client that pins its system prompt on every
+            // request): keep the newest holder and let the oldest pinned conversation go, otherwise the cache
+            // stays full of pinned entries and nothing is parked any more.
+            const auto holders = std::count_if(entries_.begin(), entries_.end(),
+                                               [](const SavedConversation& e) { return e.pinned(); });
+            if (holders <= 1) return false;
+            victim = std::find_if(entries_.begin(), entries_.end(), [](const SavedConversation& e) { return e.pinned(); });
+        }
         bytes_ -= victim->bytes();

(The comment in make_room() — "with only pinned ones left the new image does not fit" — would need the same update.)

Test added to src/core/conversation_cache_test.cpp (fails on 0.1.42 at "a fourth pinned one still parks", passes with the change; the existing pinned-prefix checks — a single pinned entry is never evicted — still pass, 4,219 checks):

    {   // every slot holds the pinned prefix: a new one evicts the oldest pinned conversation,
        // the newest holder keeps the prefix
        auto pin = [&](std::initializer_list<int32_t> ids) {
            auto c = image(ids);
            c.checkpoints.push_back(ConversationCheckpoint{});
            c.checkpoints.back().ids = {1,2};
            c.checkpoints.back().pinned = true;
            c.checkpoints.back().gdn.resize(64, 1);
            c.checkpoints.push_back(ConversationCheckpoint{});   // its own deeper checkpoint, like a real turn
            c.checkpoints.back().ids = ids;
            c.checkpoints.back().gdn.resize(64, 2);
            return c;
        };
        const size_t pb = pin({1,2,3}).bytes();
        ConversationCache full(pb * 4, 3);
        check(full.put(pin({1,2,3})) && full.put(pin({1,2,4})) && full.put(pin({1,2,5})) && full.size() == 3,
              "three pinned conversations fill the slots");
        check(full.put(pin({1,2,6})) && full.size() == 3 && full.evictions() == 1,
              "a fourth pinned one still parks: the oldest pinned conversation leaves");
        check(full.best(std::vector<int32_t>{1,2,3}, {}, true).tokens == 2, "the oldest is gone, the pinned prefix is still found");
        check(full.best(std::vector<int32_t>{1,2,6,9}, {}, true).tokens == 3, "the newest pinned conversation is found whole");
        check(full.put(image({7,8,9})) && full.size() == 3 && full.evictions() == 2, "an unpinned one also gets a slot");
    }

In use since the change (same machine and clients, ~35 min incl. a 50-request Home Assistant benchmark and a parallel 24K–56K agent conversation): no skip parking line (0 vs. 315 before); parked=32, evictions=36; 54 of 57 Home Assistant requests resumed ~10,459 of ~10,480 tokens from the cache. Short observation window, and the log does not mark which evictions were pinned ones.

Only tested with CUDA on Linux; the change is header-only and backend-independent. Happy to open a PR if that is preferred.