[FEATURE] Support Ollama request fields think and keep_alive in OllamaConfig

Abierto
  • ¿Algo no está bien?
#2,989 1 comentario 1 reacción 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
go, kubernetes, ollama, python
Área
api, backend, cloud

Línea de trabajo

Trace OllamaConfig through the CRD, adk.Ollama, and both Go and Python runtimes; review docs/kubernetes-api.md and the Ollama SDK version recorded in uv.lock. Resolve the proposed shape for think and the scope of truncate and shift, then carry optional values to both runtimes without changing requests when they are unset.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Prerequisites
Feature Summary

Let OllamaConfig set the Ollama chat request fields that are not part of options, starting with think and keep_alive.

Problem Statement / Motivation

OllamaConfig has host and options, and both runtimes forward only options from it. Ollama's ChatRequest has a few top-level fields that change how the server handles an agent's requests, and none of them can be reached through options:

Field What it does Ollama default when omitted
think Turns reasoning on or off, or sets a level. Most thinking models take true/false, gpt-oss takes low/medium/high On, for models with the thinking capability
keep_alive How long the model stays loaded after the request 5m, or the server's OLLAMA_KEEP_ALIVE
truncate Drops older history when the rendered prompt exceeds the context length true
shift Shifts the context when generation hits the limit, instead of returning an error true

So an agent on an Ollama thinking model always runs with thinking on, and there is no way to pick a reasoning level for gpt-oss. When a conversation outgrows the context length, older history is dropped without the agent being told.

Prompt caching is one place where think matters. With qwen3.6:35b-a3b, Ollama's qwen3.5 renderer puts a think block on assistant messages that come after the latest user message, and drops it once a new user message is added. The block is empty here, since the runtime does not send thinking back. In a multi-turn agent session the prompt cache therefore breaks at each new user message, and the server re-reads the previous turn's whole tool loop. In one kagent 1.0.0-alpha5 session that cost up to 36,994 tokens (364 s of prefill) for a single user message. In a direct test against the same server, the follow-up request re-read 8,086 tokens with thinking on and 49 tokens with think: false. The renderer side is tracked in ollama/ollama#18358 and ollama/ollama#17121. Turning thinking off may cost answer quality on some tasks, so it should be a per-agent choice. That is why I think it belongs in ModelConfig.

Proposed Solution

Add optional typed fields to OllamaConfig, in the same way OpenAIConfig has reasoningEffort:

apiVersion: api.kagent.dev/v1alpha3
kind: ModelConfig
spec:
  provider: Ollama
  model: qwen3.6:35b-a3b
  ollama:
    host: http://ollama.example:11434
    think: "false"    # "true", "false", "low", "medium" or "high"
    keepAlive: 30m

Carry them through adk.Ollama to both runtimes. In the Go ADK they map to ChatRequest.Think and ChatRequest.KeepAlive. In the Python ADK they map to the think and keep_alive arguments of AsyncClient.chat, which the ollama SDK version in uv.lock (0.6.2) already accepts. Fields that are not set are not sent, so existing agents behave as they do now.

Two questions before a PR:

  1. The shape of think. Ollama takes a boolean or a level string. One string enum keeps it to a single field. A boolean plus a separate level field is the other option.
  2. truncate and shift. The Go client has both fields, but the Python SDK does not expose them yet (0.6.2 chat() has no such arguments). They could go in now for the Go runtime only, or wait until the Python side can send them.
Alternatives Considered
  • A free-form passthrough map, like BedrockConfig.additionalModelRequestFields. It would cover every field at once, but docs/kubernetes-api.md asks for named, typed fields over generic extension maps.
  • Server-side defaults. Ollama has OLLAMA_KEEP_ALIVE, but no server or Modelfile setting for think, truncate or shift. On a shared server a default would also apply to every client, not one agent.
Affected Service(s)

Multiple services / System-wide (CRD, controller, Go and Python runtimes)

Additional Context

Related to #2693 and #2988, which keep the Go ADK's Ollama requests stable for prompt prefix caching.

Are you willing to contribute?
  • I am willing to submit a PR for this feature
Lenguaje dominante
Go
Estrellas
4.2k
Forks
890
Merge medio
1 d 10 h
PR fusionados (30 d)
171

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de kagent-dev/kagent

Todos los issues de kagent-dev/kagent

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.