Model-call cache barely replays on a re-run of the same head, and identical input yields disjoint findings
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 48/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- typescript
- Área
- backend, performance
Línea de trabajo
No files or tests are named. Start by tracing the model-call cache key and the run.json call and token totals, then reproduce the cold and warm runs described in the report. Done means identifying why identical packets miss the cache and making cache hit and miss counts available to callers.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
Re-reviewing the same head should be close to free: the diff is identical, the packets are identical, and only the reviewer's prior findings differ. In practice a warm run costs almost as much as a cold one.
A/B on 0.5.6, same repository, same head commit, same provider and model, both invocations with --cache, run back to back:
| run | model calls | total tokens |
|---|---|---|
| cold | 96 | 2.46M |
| warm | 80 | 2.17M |
That is 17% fewer calls and 12% fewer tokens - within the run-to-run noise of this pipeline rather than a replay.
The second signal is more interesting than the cost: the two runs produced disjoint findings on identical input. Same head, same model, same configuration, no overlap in what came back.
Questions
- What exactly is the model-call cache keyed on? If the packet prompt embeds anything that varies per run - ordering of retrieved context, timestamps, a run id, a shuffled symbol list - the key changes on every invocation and the cache can never hit on the work that matters.
- Are cache hits and misses counted anywhere a caller can read them?
run.jsonreports call and token totals, but nothing I could find states how many calls were served from the cache. Without that counter, a user cannot tell a cold pipeline from a broken cache - which is exactly the position this report is written from.
Why it matters
Review rounds repeat. A change is pushed, the reviewer runs again, and most packets are byte-identical to the previous round. If those replay, the marginal cost of an extra round approaches zero and re-reviewing becomes routine. If they do not, every round is charged at full price and teams ration the tool.
The disjoint-findings result also makes caching a correctness question, not only a cost one: if identical input can produce non-overlapping output, then a cache hit and a cache miss are not interchangeable, and it matters which one a user got.
Offer
I can re-run the A/B on 0.5.7 with any additional telemetry you want emitted, and report per-packet hit/miss if the counters are exposed.
- Lenguaje dominante
- TypeScript
- Estrellas
- 8
- Forks
- 1
- Merge medio
- 5 min
- PR fusionados (30 d)
- 2
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de 0xPolygon/codegenie
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Todos los issues de 0xPolygon/codegenie
Issues similares
-
JSON parser fails on commentsAbiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
AloisSeckar/elrh-cosca#36 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 73/100
cursor/community-plugins#572 ·
Los mantenedores suelen responder en 1 día
-
[Plugin] calendarAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
MUKE-coder/grit#228 ·
-
[BUG] Keyring unreachableAbiertobug status:needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
dongdongbh/Mindwtr#1587 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
nuxt/icon#585 · 1 comentario · 1 reacción ·
Los mantenedores suelen responder en 1 día