[Performance] Streaming persistence rewrites the full message on every delta
维护者通常 1 天内回复
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 42/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- go, sqlite
- 领域
- backend, databases, performance
调研方向
Start with pkg/runtime/persistence_observer.go and pkg/session/store.go to trace how each delta reaches the SQLite update; read pkg/runtime/observer.go to understand synchronous event delivery. Use the supplied fake-provider probe to reproduce the write growth, then add regression coverage for bounded checkpoint writes, final persisted content, completion, and cancellation. Done means persistence reduces repeated full-message writes while preserving the stated recovery behavior.
由索引模型根据 Issue 内容生成。
描述
Severity: medium · Type: performance
Location
Reviewed commit: cd65d7c776dda80d554735ee20572d09cfe2d791.
pkg/runtime/persistence_observer.go:92–102: persists each assistant text/reasoning delta.pkg/runtime/persistence_observer.go:203–229: sends the entire accumulated message to the store on every delta.pkg/session/store.go:1312–1326: marshals and replaces the fullmessage_jsonon each update.pkg/runtime/observer.go:80–84: observers run synchronously before forwarding events to the consumer.
Issue details
Incremental persistence repeatedly serializes the growing message, yielding quadratic cumulative payload volume for fixed-size deltas. The SQLite adapter executes one full-row JSON update per delta, on the event-delivery path. This buys fine-grained recovery, but long responses and reasoning streams pay for all their previous content again on every chunk.
A fake provider plus counting store exercised the actual public runtime and its stock persistence observer, with no model API requests:
| 16-byte text chunks | Final text bytes | Assistant writes (including finalization) | Cumulative text bytes passed to store | JSON bytes serialized with the same json.Marshal(msg.Message) as SQLite |
|---|---|---|---|---|
| 1,024 | 16,384 | 1,025 | 8,413,184 | 8,500,393 |
| 2,048 | 32,768 | 2,049 | 33,603,584 | 33,777,833 |
| 4,096 | 65,536 | 4,097 | 134,316,032 | 134,664,361 |
Doubling the final response roughly quadruples cumulative payload volume. These numbers measure the observer/store contract and JSON serialization volume; they are not a disk-I/O or end-to-end latency benchmark. The SQLite source establishes that each update serializes and sends the full message to an SQL update.
Save the following as /tmp/persistence-probe.go and run go run /tmp/persistence-probe.go from the repository root:
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"strings"
"github.com/docker/docker-agent/pkg/agent"
"github.com/docker/docker-agent/pkg/chat"
"github.com/docker/docker-agent/pkg/config/latest"
"github.com/docker/docker-agent/pkg/model/provider/contracts"
"github.com/docker/docker-agent/pkg/modelsdev"
"github.com/docker/docker-agent/pkg/runtime"
"github.com/docker/docker-agent/pkg/session"
"github.com/docker/docker-agent/pkg/team"
"github.com/docker/docker-agent/pkg/tools"
)
type countingStore struct {
session.Store
writes, contentBytes, jsonBytes int
}
func (s *countingStore) AddMessage(ctx context.Context, id string, msg *session.Message) (int64, error) {
s.record(msg)
return s.Store.AddMessage(ctx, id, msg)
}
func (s *countingStore) UpdateMessage(ctx context.Context, sid string, id int64, msg *session.Message) error {
s.record(msg)
return s.Store.UpdateMessage(ctx, sid, id, msg)
}
func (s *countingStore) record(msg *session.Message) {
if msg.Message.Role != chat.MessageRoleAssistant { return }
s.writes++
s.contentBytes += len(msg.Message.Content)
b, err := json.Marshal(msg.Message)
if err != nil { panic(err) }
s.jsonBytes += len(b)
}
type fakeProvider struct { n int }
func (p fakeProvider) ID() modelsdev.ID { return modelsdev.NewID("test", "mock") }
func (p fakeProvider) BaseConfig() contracts.Config {
return contracts.Config{ModelConfig: latest.ModelConfig{Provider: "test", Model: "mock"}}
}
func (p fakeProvider) CreateChatCompletionStream(context.Context, []chat.Message, []tools.Tool) (chat.MessageStream, error) {
return &fakeStream{n: p.n}, nil
}
type fakeStream struct { n, i int }
func (s *fakeStream) Recv() (chat.MessageStreamResponse, error) {
if s.i > s.n { return chat.MessageStreamResponse{}, io.EOF }
choice := chat.MessageStreamChoice{}
if s.i == s.n { choice.FinishReason = chat.FinishReasonStop } else { choice.Delta.Content = strings.Repeat("x", 16) }
s.i++
return chat.MessageStreamResponse{Choices: []chat.MessageStreamChoice{choice}}, nil
}
func (*fakeStream) Close() {}
type fakeCatalog struct{}
func (fakeCatalog) GetModel(context.Context, modelsdev.ID) (*modelsdev.Model, error) { return nil, nil }
func (fakeCatalog) GetDatabase(context.Context) (*modelsdev.Database, error) { return &modelsdev.Database{}, nil }
func main() {
for _, n := range []int{1024, 2048, 4096} {
ctx := context.Background()
store := &countingStore{Store: session.NewInMemorySessionStore()}
a := agent.New("root", "Respond.", agent.WithModel(fakeProvider{n: n}))
rt, err := runtime.New(ctx, team.New(team.WithAgents(a)), runtime.WithSessionStore(store), runtime.WithModelStore(fakeCatalog{}), runtime.WithSessionCompaction(false))
if err != nil { panic(err) }
sess := session.New(session.WithUserMessage("hi"), session.WithNonInteractive(true))
for ev := range rt.RunStream(ctx, sess) {
if e, ok := ev.(*runtime.ErrorEvent); ok { panic(e.Error) }
}
if err := rt.Close(); err != nil { panic(err) }
fmt.Printf("chunks=%d final_bytes=%d assistant_writes=%d cumulative_content_bytes=%d serialized_json_bytes=%d\n", n, n*16, store.writes, store.contentBytes, store.jsonBytes)
}
}
Impact
Large text/reasoning replies cause avoidable serialization, allocations and database updates. Because persistence is synchronous, slower storage also delays delivery to the terminal/UI and downstream observers. Concurrent sessions amplify pressure on the store.
Recommended fix
Coalesce persistence updates by time and/or added byte count, with bounded buffering and a guaranteed final flush on message completion and lifecycle teardown. Preserve partial recovery semantics explicitly. Alternatively store append-only deltas with periodic snapshots.
Add a fake-provider/counting-store regression that bounds checkpoint writes as chunk count grows, verifies the final persisted content, and covers completion/cancellation. Benchmark the SQLite-backed path before selecting checkpoint intervals.
- 主要语言
- Go
- 星标
- 4.7k
- 派生
- 565
- 平均合并
- 18 小时 52 分钟
- 30 天内合并 PR
- 221
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
docker/docker-agent 的其他 Issue
-
edit_file diff rendering panics on invalid UTF-8 text可能已有人在做 @TDnorthgarden 今天认领。 未关闭area/tui
难度 2/5 1-3 小时 新手友好度 85/100
docker/docker-agent#4955 ·
维护者通常 1 天内回复
-
A setting to turn off the models.dev catalog download (offline and strict-network use)可能已有人在做 @3431481073 于 19 天前认领。 未关闭area/config area/models
难度 2/5 1-3 小时 新手友好度 72/100
docker/docker-agent#4714 ·
维护者通常 1 天内回复
-
Error when trying to run sandbox example?可能已有人在做 @dgageot 于 28 天前认领。 未关闭area/cli
难度 2/5 1-3 小时 新手友好度 62/100
docker/docker-agent#4672 · 1 条评论 · 1 个 reaction · 已指派 1 人 ·
维护者通常 1 天内回复
-
`edit_file` silently edits the first match when `oldText` occurs more than once可能已有人在做 @dwin-gharibi 于 103 天前认领。 未关闭area/tools
难度 2/5 1-3 小时 新手友好度 78/100
docker/docker-agent#3929 · 2 条评论 ·
维护者通常 1 天内回复
-
area/api area/runtime
难度 4/5 3-5 天 新手友好度 48/100
docker/docker-agent#4951 ·
维护者通常 1 天内回复
查看 docker/docker-agent 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
IBM-Cloud/terraform-provider-ibm#7115 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 86/100
fluxcd/notification-controller#1441 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 66/100
volcengine/volcengine-go-sdk#163 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 76/100
OneBusAway/maglev#2145 ·
维护者通常 2 天内回复
-
priority/medium status/ready type/chore
难度 2/5 1-3 小时 新手友好度 84/100
discobox-ai/discobox#1357 ·
维护者通常 1 天内回复