[FEATURE] Support Ollama request fields think and keep_alive in OllamaConfig

Aperta
  • Qualcosa non va?
#2,989 1 commento 1 reazione 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
go, kubernetes, ollama, python
Ambito
api, backend, cloud

Direzione di ricerca

Trace OllamaConfig through the CRD, adk.Ollama, and both Go and Python runtimes; review docs/kubernetes-api.md and the Ollama SDK version recorded in uv.lock. Resolve the proposed shape for think and the scope of truncate and shift, then carry optional values to both runtimes without changing requests when they are unset.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Prerequisites
Feature Summary

Let OllamaConfig set the Ollama chat request fields that are not part of options, starting with think and keep_alive.

Problem Statement / Motivation

OllamaConfig has host and options, and both runtimes forward only options from it. Ollama's ChatRequest has a few top-level fields that change how the server handles an agent's requests, and none of them can be reached through options:

Field What it does Ollama default when omitted
think Turns reasoning on or off, or sets a level. Most thinking models take true/false, gpt-oss takes low/medium/high On, for models with the thinking capability
keep_alive How long the model stays loaded after the request 5m, or the server's OLLAMA_KEEP_ALIVE
truncate Drops older history when the rendered prompt exceeds the context length true
shift Shifts the context when generation hits the limit, instead of returning an error true

So an agent on an Ollama thinking model always runs with thinking on, and there is no way to pick a reasoning level for gpt-oss. When a conversation outgrows the context length, older history is dropped without the agent being told.

Prompt caching is one place where think matters. With qwen3.6:35b-a3b, Ollama's qwen3.5 renderer puts a think block on assistant messages that come after the latest user message, and drops it once a new user message is added. The block is empty here, since the runtime does not send thinking back. In a multi-turn agent session the prompt cache therefore breaks at each new user message, and the server re-reads the previous turn's whole tool loop. In one kagent 1.0.0-alpha5 session that cost up to 36,994 tokens (364 s of prefill) for a single user message. In a direct test against the same server, the follow-up request re-read 8,086 tokens with thinking on and 49 tokens with think: false. The renderer side is tracked in ollama/ollama#18358 and ollama/ollama#17121. Turning thinking off may cost answer quality on some tasks, so it should be a per-agent choice. That is why I think it belongs in ModelConfig.

Proposed Solution

Add optional typed fields to OllamaConfig, in the same way OpenAIConfig has reasoningEffort:

apiVersion: api.kagent.dev/v1alpha3
kind: ModelConfig
spec:
  provider: Ollama
  model: qwen3.6:35b-a3b
  ollama:
    host: http://ollama.example:11434
    think: "false"    # "true", "false", "low", "medium" or "high"
    keepAlive: 30m

Carry them through adk.Ollama to both runtimes. In the Go ADK they map to ChatRequest.Think and ChatRequest.KeepAlive. In the Python ADK they map to the think and keep_alive arguments of AsyncClient.chat, which the ollama SDK version in uv.lock (0.6.2) already accepts. Fields that are not set are not sent, so existing agents behave as they do now.

Two questions before a PR:

  1. The shape of think. Ollama takes a boolean or a level string. One string enum keeps it to a single field. A boolean plus a separate level field is the other option.
  2. truncate and shift. The Go client has both fields, but the Python SDK does not expose them yet (0.6.2 chat() has no such arguments). They could go in now for the Go runtime only, or wait until the Python side can send them.
Alternatives Considered
  • A free-form passthrough map, like BedrockConfig.additionalModelRequestFields. It would cover every field at once, but docs/kubernetes-api.md asks for named, typed fields over generic extension maps.
  • Server-side defaults. Ollama has OLLAMA_KEEP_ALIVE, but no server or Modelfile setting for think, truncate or shift. On a shared server a default would also apply to every client, not one agent.
Affected Service(s)

Multiple services / System-wide (CRD, controller, Go and Python runtimes)

Additional Context

Related to #2693 and #2988, which keep the Go ADK's Ollama requests stable for prompt prefix caching.

Are you willing to contribute?
  • I am willing to submit a PR for this feature
Lingua principale
Go
Stelle
4.2k
Fork
890
Merge medio
1g 10h
PR unite (30g)
171

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di kagent-dev/kagent

Tutte le issue di kagent-dev/kagent

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.