Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Server: multimodal sessions lose all KV reuse on a tail token mismatch — no common-prefix rewind, no disk-KV fallback, no continued disk stores

Abierto
  • ¿Algo no está bien?
#1,007 4 comentarios 2 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

@denovich ya está trabajando en esto.

Desde el 29/9/2026.

  • #1153 de @denovich — abierto

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
c
Área
ai-infra-agents, backend, performance

Línea de trabajo

Comienza en ds4_server.c, alrededor del fallback de live-miss, y compara el manejo de desajustes de tokens de DS4 con la ruta cercana de GLM memory-rewind. Después inspecciona las guards multimodales alrededor de kv_cache_store_current y kv_cache_try_load. Se considera terminado cuando un desajuste del token final multimodal puede reutilizar el prefijo común o la KV de disco en lugar de hacer un prefill en frío desde el token 0, y se vuelven a escribir los checkpoints.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Title: Server: multimodal sessions lose all KV reuse on a tail token mismatch — no common-prefix rewind, no disk-KV fallback, no continued disk stores

Observed on: 6289c51 (current main), DeepSeek V4 Flash Vision-Exp MXFP4, --vision server mode, --batched-session 2, disk KV enabled (--kv-disk-dir, 1 TB budget).

Summary

For multimodal (vision) chat requests, KV reuse works perfectly while the
conversation round-trips exactly (multimodal live kv hit identity=fingerprint-match,
sub-second turns on a ~335k-token prompt). But as soon as one request diverges
from the live session by even a handful of tokens at the tail, the server falls
back to a full cold prefill from token 0 — three reuse mechanisms that exist for
text sessions are all unavailable or inert for multimodal ones:

  1. No common-prefix rewind on live token-mismatch. The live-prefix check
    computes the common prefix length but only accepts it on an exact full match:

    ds4-server: live kv cache miss live=340263 prompt=340551 common=340144 vision=match reason=token-mismatch
ds4-server: chat ctx=0..340551:340551 TOOLS prompt start

    99.96% of the prompt (340,144 / 340,551 tokens) was identical to the live
    session, yet the server restarted prefill from 0. The GLM path right above
    it does exactly this kind of rewind (rewound GLM live prefix from %d to %d,
    memory-rewind cache source). The DS4 path is all-or-nothing:

    cached = common == old_pos && j->req.prompt.len >= old_pos ? common : 0;

  2. Disk KV load is gated off for multimodal requests (ds4_server.c, request
    path around the live-miss fallback):

    if (!multimodal && s->kv.enabled && cached == 0 && old_pos >= s->kv.opt.min_tokens) {
    kv_cache_store_current(s, slot, "evict");
}
if (!multimodal && cached == 0) {
    disk_cached = kv_cache_try_load(...);
}

    So a vision session that misses the live cache never consults the disk store
    at all, even when a matching checkpoint exists.

  3. No continued disk checkpoints are written during multimodal cold
    prefills.     On c0a6119 the same workload logged
    kv cache stored tokens=... reason=continued every ~30k tokens during a long
    prefill. On 6289c51, three full ~330k-token prefills (18 minutes each)
    wrote zero checkpoints — so even if (2) were reachable, there would be
    nothing fresh to load. (The text-prefix memory path did resume one later
    turn at 100,352 of 346,533 tokens, so partial reuse is not entirely dead,
    but the deep disk checkpoints are never written.)

Trigger

A coding-agent client (OpenAI chat-completions, tools, one screenshot per turn)
occasionally re-serializes the previous assistant reply slightly differently
than the server's generated token stream (thinking/DSML round-trip). The
divergence lands inside the previous reply's tail (e.g. 119 tokens before the
end of a 340,263-token history). With text sessions this costs nothing; with a
vision session every such turn is a full ~330k prefill ≈ 18 minutes on an
M3 Ultra.

Suggested fixes

  • On live token-mismatch with vision=match, rewind the DS4 session to
    common (same approach as the existing GLM memory-rewind path) instead of
    discarding the whole session.
  • Allow disk-KV load/store for multimodal sessions (store text-prefix KV even
    if image embeddings can't be serialized; the embedding cache already covers
    re-encoding cheaply).

Sanitized log excerpt

21:47:24 multimodal live kv hit images=1 cached=334535 prompt=334603 identity=fingerprint-match
21:47:25 chat ctx=334535..334603:68 TOOLS prompt done 1.041s        # ~1 s turns while round-trip is exact
...
21:55:10 live kv cache miss live=340263 prompt=340551 common=340144 vision=match reason=token-mismatch
21:55:10 chat ctx=0..340551:340551 TOOLS prompt start               # full 18-min prefill, zero continued stores

(Happy to provide longer excerpts privately if useful.)

Lenguaje dominante
C
Estrellas
24k
Forks
2.3k
Merge medio
21 d 23 h
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de antirez/ds4

Todos los issues de antirez/ds4

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.