[FEATURE] Support Ollama request fields think and keep_alive in OllamaConfig

Đang mở
  • Có gì không ổn?
#2,989 1 bình luận 1 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
48/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
go, kubernetes, ollama, python
Lĩnh vực
api, backend, cloud

Hướng nghiên cứu

Trace OllamaConfig through the CRD, adk.Ollama, and both Go and Python runtimes; review docs/kubernetes-api.md and the Ollama SDK version recorded in uv.lock. Resolve the proposed shape for think and the scope of truncate and shift, then carry optional values to both runtimes without changing requests when they are unset.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Prerequisites
Feature Summary

Let OllamaConfig set the Ollama chat request fields that are not part of options, starting with think and keep_alive.

Problem Statement / Motivation

OllamaConfig has host and options, and both runtimes forward only options from it. Ollama's ChatRequest has a few top-level fields that change how the server handles an agent's requests, and none of them can be reached through options:

Field What it does Ollama default when omitted
think Turns reasoning on or off, or sets a level. Most thinking models take true/false, gpt-oss takes low/medium/high On, for models with the thinking capability
keep_alive How long the model stays loaded after the request 5m, or the server's OLLAMA_KEEP_ALIVE
truncate Drops older history when the rendered prompt exceeds the context length true
shift Shifts the context when generation hits the limit, instead of returning an error true

So an agent on an Ollama thinking model always runs with thinking on, and there is no way to pick a reasoning level for gpt-oss. When a conversation outgrows the context length, older history is dropped without the agent being told.

Prompt caching is one place where think matters. With qwen3.6:35b-a3b, Ollama's qwen3.5 renderer puts a think block on assistant messages that come after the latest user message, and drops it once a new user message is added. The block is empty here, since the runtime does not send thinking back. In a multi-turn agent session the prompt cache therefore breaks at each new user message, and the server re-reads the previous turn's whole tool loop. In one kagent 1.0.0-alpha5 session that cost up to 36,994 tokens (364 s of prefill) for a single user message. In a direct test against the same server, the follow-up request re-read 8,086 tokens with thinking on and 49 tokens with think: false. The renderer side is tracked in ollama/ollama#18358 and ollama/ollama#17121. Turning thinking off may cost answer quality on some tasks, so it should be a per-agent choice. That is why I think it belongs in ModelConfig.

Proposed Solution

Add optional typed fields to OllamaConfig, in the same way OpenAIConfig has reasoningEffort:

apiVersion: api.kagent.dev/v1alpha3
kind: ModelConfig
spec:
  provider: Ollama
  model: qwen3.6:35b-a3b
  ollama:
    host: http://ollama.example:11434
    think: "false"    # "true", "false", "low", "medium" or "high"
    keepAlive: 30m

Carry them through adk.Ollama to both runtimes. In the Go ADK they map to ChatRequest.Think and ChatRequest.KeepAlive. In the Python ADK they map to the think and keep_alive arguments of AsyncClient.chat, which the ollama SDK version in uv.lock (0.6.2) already accepts. Fields that are not set are not sent, so existing agents behave as they do now.

Two questions before a PR:

  1. The shape of think. Ollama takes a boolean or a level string. One string enum keeps it to a single field. A boolean plus a separate level field is the other option.
  2. truncate and shift. The Go client has both fields, but the Python SDK does not expose them yet (0.6.2 chat() has no such arguments). They could go in now for the Go runtime only, or wait until the Python side can send them.
Alternatives Considered
  • A free-form passthrough map, like BedrockConfig.additionalModelRequestFields. It would cover every field at once, but docs/kubernetes-api.md asks for named, typed fields over generic extension maps.
  • Server-side defaults. Ollama has OLLAMA_KEEP_ALIVE, but no server or Modelfile setting for think, truncate or shift. On a shared server a default would also apply to every client, not one agent.
Affected Service(s)

Multiple services / System-wide (CRD, controller, Go and Python runtimes)

Additional Context

Related to #2693 and #2988, which keep the Go ADK's Ollama requests stable for prompt prefix caching.

Are you willing to contribute?
  • I am willing to submit a PR for this feature
Ngôn ngữ chính
Go
Star
4.2k
Fork
890
Merge trung bình
1 ngày 10 giờ
Pull request đã merge (30 ngày)
171

Chuẩn bị môi trường

Mở trong Codespaces

Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của kagent-dev/kagent

Tất cả issue của kagent-dev/kagent

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.