Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Server: multimodal sessions lose all KV reuse on a tail token mismatch — no common-prefix rewind, no disk-KV fallback, no continued disk stores

Aperta
  • Qualcosa non va?
#1,007 4 commenti 2 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@denovich ci sta già lavorando.

Dal 29/9/2026.

  • #1153 di @denovich — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
c
Ambito
ai-infra-agents, backend, performance

Direzione di ricerca

Inizia in ds4_server.c, nell’area del fallback di live-miss, e confronta la gestione delle discrepanze nei token DS4 con il percorso GLM memory-rewind adiacente. Poi esamina le guard multimodali attorno a kv_cache_store_current e kv_cache_try_load. Il lavoro è completo quando una discrepanza del token finale multimodale può riutilizzare il prefisso comune o la KV su disco invece di eseguire un prefill a freddo dal token 0, e i checkpoint vengono scritti nuovamente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Title: Server: multimodal sessions lose all KV reuse on a tail token mismatch — no common-prefix rewind, no disk-KV fallback, no continued disk stores

Observed on: 6289c51 (current main), DeepSeek V4 Flash Vision-Exp MXFP4, --vision server mode, --batched-session 2, disk KV enabled (--kv-disk-dir, 1 TB budget).

Summary

For multimodal (vision) chat requests, KV reuse works perfectly while the
conversation round-trips exactly (multimodal live kv hit identity=fingerprint-match,
sub-second turns on a ~335k-token prompt). But as soon as one request diverges
from the live session by even a handful of tokens at the tail, the server falls
back to a full cold prefill from token 0 — three reuse mechanisms that exist for
text sessions are all unavailable or inert for multimodal ones:

  1. No common-prefix rewind on live token-mismatch. The live-prefix check
    computes the common prefix length but only accepts it on an exact full match:

    ds4-server: live kv cache miss live=340263 prompt=340551 common=340144 vision=match reason=token-mismatch
ds4-server: chat ctx=0..340551:340551 TOOLS prompt start

    99.96% of the prompt (340,144 / 340,551 tokens) was identical to the live
    session, yet the server restarted prefill from 0. The GLM path right above
    it does exactly this kind of rewind (rewound GLM live prefix from %d to %d,
    memory-rewind cache source). The DS4 path is all-or-nothing:

    cached = common == old_pos && j->req.prompt.len >= old_pos ? common : 0;

  2. Disk KV load is gated off for multimodal requests (ds4_server.c, request
    path around the live-miss fallback):

    if (!multimodal && s->kv.enabled && cached == 0 && old_pos >= s->kv.opt.min_tokens) {
    kv_cache_store_current(s, slot, "evict");
}
if (!multimodal && cached == 0) {
    disk_cached = kv_cache_try_load(...);
}

    So a vision session that misses the live cache never consults the disk store
    at all, even when a matching checkpoint exists.

  3. No continued disk checkpoints are written during multimodal cold
    prefills.     On c0a6119 the same workload logged
    kv cache stored tokens=... reason=continued every ~30k tokens during a long
    prefill. On 6289c51, three full ~330k-token prefills (18 minutes each)
    wrote zero checkpoints — so even if (2) were reachable, there would be
    nothing fresh to load. (The text-prefix memory path did resume one later
    turn at 100,352 of 346,533 tokens, so partial reuse is not entirely dead,
    but the deep disk checkpoints are never written.)

Trigger

A coding-agent client (OpenAI chat-completions, tools, one screenshot per turn)
occasionally re-serializes the previous assistant reply slightly differently
than the server's generated token stream (thinking/DSML round-trip). The
divergence lands inside the previous reply's tail (e.g. 119 tokens before the
end of a 340,263-token history). With text sessions this costs nothing; with a
vision session every such turn is a full ~330k prefill ≈ 18 minutes on an
M3 Ultra.

Suggested fixes

  • On live token-mismatch with vision=match, rewind the DS4 session to
    common (same approach as the existing GLM memory-rewind path) instead of
    discarding the whole session.
  • Allow disk-KV load/store for multimodal sessions (store text-prefix KV even
    if image embeddings can't be serialized; the embedding cache already covers
    re-encoding cheaply).

Sanitized log excerpt

21:47:24 multimodal live kv hit images=1 cached=334535 prompt=334603 identity=fingerprint-match
21:47:25 chat ctx=334535..334603:68 TOOLS prompt done 1.041s        # ~1 s turns while round-trip is exact
...
21:55:10 live kv cache miss live=340263 prompt=340551 common=340144 vision=match reason=token-mismatch
21:55:10 chat ctx=0..340551:340551 TOOLS prompt start               # full 18-min prefill, zero continued stores

(Happy to provide longer excerpts privately if useful.)

Lingua principale
C
Stelle
24k
Fork
2.3k
Merge medio
21g 23h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di antirez/ds4

Tutte le issue di antirez/ds4

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.