Core LLM layer resends the full conversation every turn — no opt-in server-side conversation reuse (`previous_response_id`) on the Responses API

Aperta
#6,690 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
39/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
ai, backend

Direzione di ricerca

Start by reading _build_responses_params and _build_messages in src/praisonai-agents/praisonaiagents/llm/llm.py, then trace the sync, sync-stream, and async Responses API call paths. The work needs to track response IDs across turns and handle opt-in configuration, provider support, and invalidated history. It is done when enabled sessions reuse server-side state with safe fallbacks, while the default behavior remains stateless.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug claude

Summary

On the OpenAI Responses API path, every agent turn resends the entire conversation as input. There is no option to reuse the provider's own server-side conversation state (previous_response_id + store=True), so input tokens and request latency grow with the length of a session.

This is most costly for the SDK's long-lived surfaces — a gateway/bot channel session can run for hundreds of turns, and each turn re-uploads the whole (or compacted) history. Local context compaction shrinks the payload but still resends it on every call. A provider that already persists prior turns is being asked to re-ingest them each time.

Adding an opt-in server-side conversation mode (default behaviour unchanged) would cut input tokens to the per-turn delta, lower latency, and keep the SDK's long-running sessions efficient — without affecting providers or configurations that don't support it.

Current behaviour

_build_responses_params always flattens the full message list into input with no continuation handle:

# src/praisonai-agents/praisonaiagents/llm/llm.py  (_build_responses_params)
# system message  → instructions
# remaining messages  → input (list of role/content items)
# ...
params: Dict[str, Any] = {"model": self._resolve_openai_compatible_model()}
# ... builds instructions + input_items from EVERY message, every call ...
# no `previous_response_id`, no `store`

The message list itself is rebuilt in full each turn:

# src/praisonai-agents/praisonaiagents/llm/llm.py
def _build_messages(self, prompt, system_prompt=None, chat_history=None, ...):
    # assembles system_prompt + full chat_history + prompt every turn

The Responses API is actively used on three call sites (_build_responses_params is invoked from the sync loop, sync-stream loop, and async loop), confirming this is the live path for OpenAI reasoning / gpt-5 / o-series models:

# src/praisonai-agents/praisonaiagents/llm/llm.py
if self._supports_responses_api():
    responses_params = self._build_responses_params(...)
    resp = self._call_responses_api(**responses_params)

Grep confirms there is no previous_response_id, store=True, or conversation-continuation handle anywhere under praisonaiagents/llm/.

Note: this is distinct from the already-addressed gateway inbound /v1/responses HTTP surface (persistence/idempotency, e.g. #5335, #5726). This issue is about the agent's outbound calls to the model provider.

Desired behaviour

An opt-in mode where, on providers that persist server-side conversation state, the SDK:

  • sends store=True and captures the returned response.id,
  • on the next turn sends only the new input items plus previous_response_id=<last id>, instead of the full history,
  • transparently falls back to full-history resend when the provider/model doesn't support it, when state is lost, or when local history is mutated (compaction, /compress, history edits) in a way that invalidates the server-side chain.

Default remains the current stateless, fully-portable behaviour. The feature engages only when explicitly enabled and the provider supports it.

Layer placement

  • Primary layer: core (src/praisonai-agents/praisonaiagents/llm/)
  • Why not core: n/a — the resend happens in core's provider call construction (_build_responses_params), and the continuation handle must be tracked across turns inside the core LLM/session loop. This is the only place with the request/response pair.
  • Why not wrapper: the wrapper can't inject previous_response_id without reaching into core's per-turn request building and response handling; it only exposes config.
  • Why not tools: this is model-call plumbing, not an agent-callable integration.
  • Why not plugins: it isn't a lifecycle hook/guardrail; it changes how the core LLM request is constructed and how response ids are threaded between turns.
  • Secondary touch (optional): ExecutionConfig flag (e.g. server_side_conversation), plus a wrapper CLI/YAML surface so it's configurable the 3 standard ways.
  • 3-way surface (CLI + YAML + Python): yes — Python (ExecutionConfig/Agent(...)), YAML (agents.yaml), CLI flag.

Proposed approach

  • Extension point: ExecutionConfig option + a small continuation-state field threaded through the LLM turn loop; gated behind a self._supports_responses_api()-style capability probe.
  • Minimal API sketch:
# opt-in; default False preserves today's stateless behaviour
agent = Agent(
    name="assistant",
    llm="gpt-5",
    server_side_conversation=True,   # or via ExecutionConfig / YAML / CLI
)

Resolution sketch

# Before (today) — src/praisonai-agents/praisonaiagents/llm/llm.py
def _build_responses_params(self, messages, tools=None, temperature=None, **kwargs):
    params = {"model": self._resolve_openai_compatible_model()}
    instructions, input_items = self._split_system_and_input(messages)  # ALL messages, every turn
    params["input"] = input_items
    return params

# After (proposed) — opt-in, backward compatible
def _build_responses_params(self, messages, tools=None, temperature=None, **kwargs):
    params = {"model": self._resolve_openai_compatible_model()}
    if self._server_side_conversation_active():        # opt-in AND provider supports it
        prev_id = self._last_response_id               # captured from previous turn
        if prev_id and not self._history_was_mutated(): # compaction/edit invalidates the chain
            params["store"] = True
            params["previous_response_id"] = prev_id
            params["input"] = self._delta_input_items(messages)  # only NEW items since prev_id
            return params
        params["store"] = True                         # start/refresh the chain
    instructions, input_items = self._split_system_and_input(messages)  # full resend (fallback/default)
    params["input"] = input_items
    return params

# and, after each call, capture the id to chain the next turn:
#   self._last_response_id = resp.id

Severity

Medium — not a correctness bug; it's an opt-in cost/latency optimisation. But it compounds on the SDK's primary long-running workload (gateway channel sessions), where per-turn input currently grows O(turns); server-side chaining makes each request O(delta).

Validation

  • Read src/praisonai-agents/praisonaiagents/llm/llm.py _build_responses_params in full: it maps the complete message list to input on every call, with no store / previous_response_id.
  • Read _build_messages: full system + chat_history + prompt reassembled each turn.
  • Confirmed the Responses API path is live (3 invocation sites: sync, sync-stream, async) behind _supports_responses_api().
  • Grep across praisonaiagents/llm/ for previous_response_id / store=True / conversation-continuation: no matches.
  • Checked existing issues: related closed items (#5335, #5726) concern the gateway's inbound /v1/responses HTTP surface, not the agent's outbound provider calls — no duplicate.
Lingua principale
Python
Stelle
9.3k
Fork
1.5k
Merge medio
1g 5h
PR unite (30g)
520

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di MervinPraison/PraisonAI

Tutte le issue di MervinPraison/PraisonAI

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.