[FEATURE] Support Ollama request fields think and keep_alive in OllamaConfig

未关闭
  • 哪里不对？
#2,989 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
go, kubernetes, ollama, python
领域
api, backend, cloud

调研方向

Trace OllamaConfig through the CRD, adk.Ollama, and both Go and Python runtimes; review docs/kubernetes-api.md and the Ollama SDK version recorded in uv.lock. Resolve the proposed shape for think and the scope of truncate and shift, then carry optional values to both runtimes without changing requests when they are unset.

由索引模型根据 Issue 内容生成。

描述

Prerequisites
Feature Summary

Let OllamaConfig set the Ollama chat request fields that are not part of options, starting with think and keep_alive.

Problem Statement / Motivation

OllamaConfig has host and options, and both runtimes forward only options from it. Ollama's ChatRequest has a few top-level fields that change how the server handles an agent's requests, and none of them can be reached through options:

Field What it does Ollama default when omitted
think Turns reasoning on or off, or sets a level. Most thinking models take true/false, gpt-oss takes low/medium/high On, for models with the thinking capability
keep_alive How long the model stays loaded after the request 5m, or the server's OLLAMA_KEEP_ALIVE
truncate Drops older history when the rendered prompt exceeds the context length true
shift Shifts the context when generation hits the limit, instead of returning an error true

So an agent on an Ollama thinking model always runs with thinking on, and there is no way to pick a reasoning level for gpt-oss. When a conversation outgrows the context length, older history is dropped without the agent being told.

Prompt caching is one place where think matters. With qwen3.6:35b-a3b, Ollama's qwen3.5 renderer puts a think block on assistant messages that come after the latest user message, and drops it once a new user message is added. The block is empty here, since the runtime does not send thinking back. In a multi-turn agent session the prompt cache therefore breaks at each new user message, and the server re-reads the previous turn's whole tool loop. In one kagent 1.0.0-alpha5 session that cost up to 36,994 tokens (364 s of prefill) for a single user message. In a direct test against the same server, the follow-up request re-read 8,086 tokens with thinking on and 49 tokens with think: false. The renderer side is tracked in ollama/ollama#18358 and ollama/ollama#17121. Turning thinking off may cost answer quality on some tasks, so it should be a per-agent choice. That is why I think it belongs in ModelConfig.

Proposed Solution

Add optional typed fields to OllamaConfig, in the same way OpenAIConfig has reasoningEffort:

apiVersion: api.kagent.dev/v1alpha3
kind: ModelConfig
spec:
  provider: Ollama
  model: qwen3.6:35b-a3b
  ollama:
    host: http://ollama.example:11434
    think: "false"    # "true", "false", "low", "medium" or "high"
    keepAlive: 30m

Carry them through adk.Ollama to both runtimes. In the Go ADK they map to ChatRequest.Think and ChatRequest.KeepAlive. In the Python ADK they map to the think and keep_alive arguments of AsyncClient.chat, which the ollama SDK version in uv.lock (0.6.2) already accepts. Fields that are not set are not sent, so existing agents behave as they do now.

Two questions before a PR:

  1. The shape of think. Ollama takes a boolean or a level string. One string enum keeps it to a single field. A boolean plus a separate level field is the other option.
  2. truncate and shift. The Go client has both fields, but the Python SDK does not expose them yet (0.6.2 chat() has no such arguments). They could go in now for the Go runtime only, or wait until the Python side can send them.
Alternatives Considered
  • A free-form passthrough map, like BedrockConfig.additionalModelRequestFields. It would cover every field at once, but docs/kubernetes-api.md asks for named, typed fields over generic extension maps.
  • Server-side defaults. Ollama has OLLAMA_KEEP_ALIVE, but no server or Modelfile setting for think, truncate or shift. On a shared server a default would also apply to every client, not one agent.
Affected Service(s)

Multiple services / System-wide (CRD, controller, Go and Python runtimes)

Additional Context

Related to #2693 and #2988, which keep the Go ADK's Ollama requests stable for prompt prefix caching.

Are you willing to contribute?
  • I am willing to submit a PR for this feature
主要语言
Go
星标
4.2k
派生
890
平均合并
1 天 10 小时
30 天内合并 PR
171

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

kagent-dev/kagent 的其他 Issue

查看 kagent-dev/kagent 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。