conversation cache: once every slot holds a pinned prefix, nothing is parked any more ("skip parking ... exceeds available budget")

Start in include/strata/core/conversation_cache.hpp at evict_oldest() and its caller make_room(), then read the test block in src/core/conversation_cache_test.cpp that the issue quotes. Apply the proposed change so that a pinned victim is evicted while another pin holder remains, and update the make_room() comment. Done when the new "fourth pinned one still parks" check passes and the existing pinned-prefix checks still pass.

Version: 0.1.42 (single RTX 3090, Linux, CUDA). evict_oldest() is unchanged on main (a0116b27, 2026-10-12).

Setup: Home Assistant's voice assistant talks to Strata through an OpenAI-compatible gateway that adds strata_prefix to every request, so the system prompt + tool list (~6–10K tokens) stay pinned. --conversation-cache-mib 22528 --conversation-cache-slots 32, "parallel": 2. Other clients (agents, long-context jobs) share the engine.

What happens: every Home Assistant conversation carries the pinned checkpoint in its chain, so SavedConversation::pinned() is true for all of them. make_room() → evict_oldest() only evicts entries that are not pinned. Once all 32 slots held pinned conversations, evict_oldest() returned false and every attempt to park was refused:

strata serve: conversation cache: skip parking (snapshot 451 MiB exceeds available budget)

(315 times in the ~2,000 log lines after the last successful park, no park in between, snapshots from 225 to 3,806 MiB alike, while the parked bytes were below the 22 GiB budget and ~11 GiB of RAM were still available — so the slot count with only pinned entries, not the byte budget, refused them). The last successful park before that read parked=32 ... evictions=309. Effects: a conversation only resumed from the cache while it was still the active one; everything else was read again — ~4 s instead of ~0.4–0.8 s for a 7K voice request, and three long-context requests (186K–202K tokens) were read from token 0 (~90–98 s each).

Relation to #1559: #1559 retires stale pins when a new prefix replaces the old one. In our log the pin position changed several times (4578, 5751, 5970, 7250, 6484 tokens — the Home Assistant prompt changes when entities are renamed or exposed), so stale pins probably contributed, and #1559 may already fix part of what we saw; I have not run with #1559 applied. What #1559 cannot cover: a client with a stable prefix. Every parked conversation then holds the current pin, #1559 keeps them all pinned, and once slots conversations are parked evict_oldest() still returns false. This patch is for that case and is independent of #1559.

Proposed fix (minimal): when only pinned entries are left, evict the oldest pinned one as long as another holder of a pin remains (the newest is kept), so the shared prefix stays parked and findable. Note that make_room() calls evict_oldest() in a loop, so a very large incoming snapshot can clear all pinned entries but the newest one, not just one.

--- a/include/strata/core/conversation_cache.hpp
+++ b/include/strata/core/conversation_cache.hpp
@@
-    // False when none can go (empty, or only entries that hold a pinned shared prefix are left).
+    // False when none can go (empty, or a single entry that holds a pinned shared prefix is left).
     bool evict_oldest() {
         auto victim = std::find_if(entries_.begin(), entries_.end(), [](const SavedConversation& e) { return !e.pinned(); });
-        if (victim == entries_.end()) return false;
+        if (victim == entries_.end()) {
+            // Every parked conversation holds a pinned prefix (a client that pins its system prompt on every
+            // request): keep the newest holder and let the oldest pinned conversation go, otherwise the cache
+            // stays full of pinned entries and nothing is parked any more.
+            const auto holders = std::count_if(entries_.begin(), entries_.end(),
+                                               [](const SavedConversation& e) { return e.pinned(); });
+            if (holders <= 1) return false;
+            victim = std::find_if(entries_.begin(), entries_.end(), [](const SavedConversation& e) { return e.pinned(); });
+        }
         bytes_ -= victim->bytes();

(The comment in make_room() — "with only pinned ones left the new image does not fit" — would need the same update.)

Test added to src/core/conversation_cache_test.cpp (fails on 0.1.42 at "a fourth pinned one still parks", passes with the change; the existing pinned-prefix checks — a single pinned entry is never evicted — still pass, 4,219 checks):

    {   // every slot holds the pinned prefix: a new one evicts the oldest pinned conversation,
        // the newest holder keeps the prefix
        auto pin = [&](std::initializer_list<int32_t> ids) {
            auto c = image(ids);
            c.checkpoints.push_back(ConversationCheckpoint{});
            c.checkpoints.back().ids = {1,2};
            c.checkpoints.back().pinned = true;
            c.checkpoints.back().gdn.resize(64, 1);
            c.checkpoints.push_back(ConversationCheckpoint{});   // its own deeper checkpoint, like a real turn
            c.checkpoints.back().ids = ids;
            c.checkpoints.back().gdn.resize(64, 2);
            return c;
        };
        const size_t pb = pin({1,2,3}).bytes();
        ConversationCache full(pb * 4, 3);
        check(full.put(pin({1,2,3})) && full.put(pin({1,2,4})) && full.put(pin({1,2,5})) && full.size() == 3,
              "three pinned conversations fill the slots");
        check(full.put(pin({1,2,6})) && full.size() == 3 && full.evictions() == 1,
              "a fourth pinned one still parks: the oldest pinned conversation leaves");
        check(full.best(std::vector<int32_t>{1,2,3}, {}, true).tokens == 2, "the oldest is gone, the pinned prefix is still found");
        check(full.best(std::vector<int32_t>{1,2,6,9}, {}, true).tokens == 3, "the newest pinned conversation is found whole");
        check(full.put(image({7,8,9})) && full.size() == 3 && full.evictions() == 2, "an unpinned one also gets a slot");
    }

In use since the change (same machine and clients, ~35 min incl. a 50-request Home Assistant benchmark and a parallel 24K–56K agent conversation): no skip parking line (0 vs. 315 before); parked=32, evictions=36; 54 of 57 Home Assistant requests resumed ~10,459 of ~10,480 tokens from the cache. Short observation window, and the log does not mark which evictions were pinned ones.

Only tested with CUDA on Linux; the change is header-only and backend-independent. Happy to open a PR if that is preferred.