Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Keep prompt prefix stable in the chat path (cache-friendly conversation history)

Aperta
  • Qualcosa non va?
#268 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
15/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

This is design work across the chat path. Start with nc_model.py, where history truncation and cutoff rewrite already-sent messages, and find where the system prompt, tool list and task usage are assembled per request. Done would need a maintainer-agreed design for a pinned system prompt, a stable tool order, append-only truncation, and cache-token fields in usage. Settle the design with maintainers before writing code.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

Feature request

Disclaimer

Initial spec elicited via STT whisper-base, partially polished with opencode:GLM-5.3 post-explore.

Is your feature request related to a problem? Please describe.

Prompt-cache hit rate on the context_agent chat path is degraded because the request prefix mutates during a conversation, which raises cost and latency on every long chat. All major providers (OpenAI, Anthropic, Gemini) cache by prefix match, so any mutation re-bills everything after it as uncached input.

Mutations on main:

  • System prompt recomposed per task: current date, conditional tool lines, memories, skills list.
  • Tool output truncation and history cutoff in nc_model.py rewrite already-sent history.
  • Tool list/ordering rebuilt per request.

Describe the solution you'd like

  • Deterministic system prompt per conversation; updates (date, memories) go in a reminder block in the next user message instead (Anthropic: mid-conversation system/tool changes).
  • Stable tool list and ordering for the session's lifetime (tool use with prompt caching).
  • Append-only history: large tool outputs truncated once when first sent, never re-truncated as thresholds shift, keeps the #91 payload-cap fix without rewriting the prefix.
  • Expose cache hits in task usage (cached_tokens / cache_creation_input_tokens) so regressions are measurable.

Describe alternatives you've considered

Doing nothing: caching still partially works, but every prefix mutation re-bills the whole conversation as uncached input.

Additional context / related

  • nextcloud/assistant#698
    • conversation branching; per-branch caches depend on these guarantees.
  • #246
    • configurable system prompt; must also be pinned per conversation, same code path.
  • #202 / #247
    • tool-set management; mid-session tool changes are themselves prefix invalidations, need a session-stable or deferred-loading design.
  • #213
    • compaction should use cache-safe forking (identical prefix, summary appended) or it defeats this.
  • #91 (closed)
    • TaskProcessing payload cap that motivated retroactive truncation; constraint on the append-only approach.
  • reference
Lingua principale
Python
Stelle
26
Fork
20
Merge medio
2g 9h
PR unite (30g)
4

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di nextcloud/context_agent

Tutte le issue di nextcloud/context_agent

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.