[Performance] Streaming persistence rewrites the full message on every delta
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 42/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Stack tecnológico
- go, sqlite
- Área
- backend, databases, performance
Línea de trabajo
Start with pkg/runtime/persistence_observer.go and pkg/session/store.go to trace how each delta reaches the SQLite update; read pkg/runtime/observer.go to understand synchronous event delivery. Use the supplied fake-provider probe to reproduce the write growth, then add regression coverage for bounded checkpoint writes, final persisted content, completion, and cancellation. Done means persistence reduces repeated full-message writes while preserving the stated recovery behavior.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Severity: medium · Type: performance
Location
Reviewed commit: cd65d7c776dda80d554735ee20572d09cfe2d791.
pkg/runtime/persistence_observer.go:92–102: persists each assistant text/reasoning delta.pkg/runtime/persistence_observer.go:203–229: sends the entire accumulated message to the store on every delta.pkg/session/store.go:1312–1326: marshals and replaces the fullmessage_jsonon each update.pkg/runtime/observer.go:80–84: observers run synchronously before forwarding events to the consumer.
Issue details
Incremental persistence repeatedly serializes the growing message, yielding quadratic cumulative payload volume for fixed-size deltas. The SQLite adapter executes one full-row JSON update per delta, on the event-delivery path. This buys fine-grained recovery, but long responses and reasoning streams pay for all their previous content again on every chunk.
A fake provider plus counting store exercised the actual public runtime and its stock persistence observer, with no model API requests:
| 16-byte text chunks | Final text bytes | Assistant writes (including finalization) | Cumulative text bytes passed to store | JSON bytes serialized with the same json.Marshal(msg.Message) as SQLite |
|---|---|---|---|---|
| 1,024 | 16,384 | 1,025 | 8,413,184 | 8,500,393 |
| 2,048 | 32,768 | 2,049 | 33,603,584 | 33,777,833 |
| 4,096 | 65,536 | 4,097 | 134,316,032 | 134,664,361 |
Doubling the final response roughly quadruples cumulative payload volume. These numbers measure the observer/store contract and JSON serialization volume; they are not a disk-I/O or end-to-end latency benchmark. The SQLite source establishes that each update serializes and sends the full message to an SQL update.
Save the following as /tmp/persistence-probe.go and run go run /tmp/persistence-probe.go from the repository root:
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"strings"
"github.com/docker/docker-agent/pkg/agent"
"github.com/docker/docker-agent/pkg/chat"
"github.com/docker/docker-agent/pkg/config/latest"
"github.com/docker/docker-agent/pkg/model/provider/contracts"
"github.com/docker/docker-agent/pkg/modelsdev"
"github.com/docker/docker-agent/pkg/runtime"
"github.com/docker/docker-agent/pkg/session"
"github.com/docker/docker-agent/pkg/team"
"github.com/docker/docker-agent/pkg/tools"
)
type countingStore struct {
session.Store
writes, contentBytes, jsonBytes int
}
func (s *countingStore) AddMessage(ctx context.Context, id string, msg *session.Message) (int64, error) {
s.record(msg)
return s.Store.AddMessage(ctx, id, msg)
}
func (s *countingStore) UpdateMessage(ctx context.Context, sid string, id int64, msg *session.Message) error {
s.record(msg)
return s.Store.UpdateMessage(ctx, sid, id, msg)
}
func (s *countingStore) record(msg *session.Message) {
if msg.Message.Role != chat.MessageRoleAssistant { return }
s.writes++
s.contentBytes += len(msg.Message.Content)
b, err := json.Marshal(msg.Message)
if err != nil { panic(err) }
s.jsonBytes += len(b)
}
type fakeProvider struct { n int }
func (p fakeProvider) ID() modelsdev.ID { return modelsdev.NewID("test", "mock") }
func (p fakeProvider) BaseConfig() contracts.Config {
return contracts.Config{ModelConfig: latest.ModelConfig{Provider: "test", Model: "mock"}}
}
func (p fakeProvider) CreateChatCompletionStream(context.Context, []chat.Message, []tools.Tool) (chat.MessageStream, error) {
return &fakeStream{n: p.n}, nil
}
type fakeStream struct { n, i int }
func (s *fakeStream) Recv() (chat.MessageStreamResponse, error) {
if s.i > s.n { return chat.MessageStreamResponse{}, io.EOF }
choice := chat.MessageStreamChoice{}
if s.i == s.n { choice.FinishReason = chat.FinishReasonStop } else { choice.Delta.Content = strings.Repeat("x", 16) }
s.i++
return chat.MessageStreamResponse{Choices: []chat.MessageStreamChoice{choice}}, nil
}
func (*fakeStream) Close() {}
type fakeCatalog struct{}
func (fakeCatalog) GetModel(context.Context, modelsdev.ID) (*modelsdev.Model, error) { return nil, nil }
func (fakeCatalog) GetDatabase(context.Context) (*modelsdev.Database, error) { return &modelsdev.Database{}, nil }
func main() {
for _, n := range []int{1024, 2048, 4096} {
ctx := context.Background()
store := &countingStore{Store: session.NewInMemorySessionStore()}
a := agent.New("root", "Respond.", agent.WithModel(fakeProvider{n: n}))
rt, err := runtime.New(ctx, team.New(team.WithAgents(a)), runtime.WithSessionStore(store), runtime.WithModelStore(fakeCatalog{}), runtime.WithSessionCompaction(false))
if err != nil { panic(err) }
sess := session.New(session.WithUserMessage("hi"), session.WithNonInteractive(true))
for ev := range rt.RunStream(ctx, sess) {
if e, ok := ev.(*runtime.ErrorEvent); ok { panic(e.Error) }
}
if err := rt.Close(); err != nil { panic(err) }
fmt.Printf("chunks=%d final_bytes=%d assistant_writes=%d cumulative_content_bytes=%d serialized_json_bytes=%d\n", n, n*16, store.writes, store.contentBytes, store.jsonBytes)
}
}
Impact
Large text/reasoning replies cause avoidable serialization, allocations and database updates. Because persistence is synchronous, slower storage also delays delivery to the terminal/UI and downstream observers. Concurrent sessions amplify pressure on the store.
Recommended fix
Coalesce persistence updates by time and/or added byte count, with bounded buffering and a guaranteed final flush on message completion and lifecycle teardown. Preserve partial recovery semantics explicitly. Alternatively store append-only deltas with periodic snapshots.
Add a fake-provider/counting-store regression that bounds checkpoint writes as chunk count grows, verifies the final persisted content, and covers completion/cancellation. Benchmark the SQLite-backed path before selecting checkpoint intervals.
- Lenguaje dominante
- Go
- Estrellas
- 4.7k
- Forks
- 565
- Merge medio
- 18 h 52 min
- PR fusionados (30 d)
- 221
Preparar el entorno
Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.
- Incluye un Dockerfile o un archivo de Docker Compose
- Sin plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de docker/docker-agent
-
edit_file diff rendering panics on invalid UTF-8 textPosiblemente ocupada @TDnorthgarden la tomó hoy. Abiertoarea/tui
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
docker/docker-agent#4955 ·
Los mantenedores suelen responder en 1 día
-
A setting to turn off the models.dev catalog download (offline and strict-network use)Posiblemente ocupada @3431481073 la tomó hace 19 días. Abiertoarea/config area/models
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
docker/docker-agent#4714 ·
Los mantenedores suelen responder en 1 día
-
Error when trying to run sandbox example?Posiblemente ocupada @dgageot la tomó hace 28 días. Abiertoarea/cli
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
docker/docker-agent#4672 · 1 comentario · 1 reacción · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
`edit_file` silently edits the first match when `oldText` occurs more than oncePosiblemente ocupada @dwin-gharibi la tomó hace 103 días. Abiertoarea/tools
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
docker/docker-agent#3929 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
-
area/api area/runtime
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
docker/docker-agent#4951 ·
Los mantenedores suelen responder en 1 día
Todos los issues de docker/docker-agent
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
IBM-Cloud/terraform-provider-ibm#7115 ·
Los mantenedores suelen responder en 2 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
fluxcd/notification-controller#1441 ·
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100
volcengine/volcengine-go-sdk#163 · 1 comentario ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
OneBusAway/maglev#2145 ·
Los mantenedores suelen responder en 2 días
-
priority/medium status/ready type/chore
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
discobox-ai/discobox#1357 ·
Los mantenedores suelen responder en 1 día