billing: interrupted stream retry can omit potentially billed attempt from session cost

Aperta
#65,361 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
32/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
sqlite, typescript
Ambito
backend

Direzione di ricerca

Start from packages/core/src/session/runner/step.ts around the retry and usage paths, then read message-updater.ts where the next step clears retry state. Run the bundled repro.py against the local mock to confirm the session row shows one $0.04 step while the mock ledger records two. Done means the interrupted attempt is preserved with explicit unknown cost and retry attribution, covered by a regression test at the session accounting layer.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area/core platform/windows type/feature version/v2

Description

When an OpenAI-compatible streaming response is interrupted before its final usage arrives, OpenCode 2.3.2 can retry it using the same assistant message. The session cost then contains the successful retry's cost but no cost or uncertainty for the failed attempt. If a provider charges for the first attempt, the displayed session total can understate the actual charge.

This is a conditional accounting/visibility issue, not a claim that any particular provider charged for a failed request. No real provider is contacted in the reproduction below: its server deliberately records a fake $0.04 charge per request.

Plugins

No user plugins configured in the isolated test. A fresh temporary home, config and database are used; OpenCode's built-in plugins remain in their default state.

OpenCode version

2.3.2 (opencode v2.3.2, Windows Desktop bundled CLI, standalone mode).

Steps to reproduce

  1. On Windows with Python 3 and OpenCode 2.3.2, save the script below as repro.py. If opencode is not on PATH, set OPENCODE_CLI to the CLI executable's absolute path.
  2. Run python repro.py. It launches an OpenAI-compatible server bound to 127.0.0.1, configures only a fake local model and key, and uses a separate temporary OpenCode database. It runs a normal control and an interrupted-stream scenario.
  3. Compare the fake server ledger with the session cost. In the interrupted case the server logs a fake $0.04 charge, sends part of a chunked response and resets the connection before the final usage event. The retry succeeds with a second fake $0.04 charge and reports that usage.

Observed twice with the local test harness; the self-contained script below was then run once and checks the outcome with assertions:

{"mode":"control","mock_attempts":1,"mock_total":0.04,"session_total":0.04,"assistant_steps":1,"retry_errors":[]}
{"mode":"disconnect","mock_attempts":2,"mock_total":0.08,"session_total":0.04,"assistant_steps":1,"retry_errors":["provider.transport"]}

In the failing case, live CLI JSON reports provider.transport with ECONNRESET and retry attempt 2. The database contains one successful assistant step with $0.04 and no separate charge for the first attempt.

Expected: Do not imply the session total is complete when a retry can incur an unknown charge; preserve attempt/failure attribution and clearly surface unknown cost, or reconcile actual provider-reported charges when available. The client cannot infer an unreported charge solely from a broken stream.

Actual: The session presents $0.04 even though the controlled mock ledger records $0.08, with the retry absent from the final assistant-step count. The simulated ledger is an explicit assumption of this test, not independent proof of real billing behavior.

Screenshot

Not applicable. Script and output below contain only synthetic data.

Operating System

Windows 11 x64, build 10.0.26300.10075.

Terminal

Noninteractive standalone CLI launched by Python from PowerShell. COMSPEC=cmd.exe.

Additional context

In v2.3.2 step.ts, a retry before output may reuse the same assistant message. In message-updater.ts, the next step start clears retry state; step.ts derives saved usage from a finish record. Whether a real provider charges a disconnected request and whether usage can be recovered are separate questions.

Related to #65243 (an account-specific support issue, not a proven instance of this cause) and #63293 (a different provider's metadata persistence).

Self-contained local reproduction script
"""Offline reproduction for OpenCode v2.3.2 on Windows; no real API credentials.

Set OPENCODE_CLI to the full path to opencode.exe if it is not on PATH, then run
`python opencode-retry-repro.py`. Both scenarios use independent temporary data.
The $0.04/request in this script is a deliberately simulated ledger entry.
"""
import json
import os
import shutil
import socket
import sqlite3
import struct
import subprocess
import tempfile
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path

CLI = os.environ.get("OPENCODE_CLI") or shutil.which("opencode")
if not CLI:
    raise SystemExit("Set OPENCODE_CLI to the OpenCode CLI executable")
CLI = str(Path(CLI).resolve())


class MockHandler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def log_message(self, *_):
        pass

    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers.get("Content-Length", "0"))))
        if self.path != "/v1/chat/completions" or body.get("model") != "mock-model":
            self.send_error(400)
            return
        with self.server.lock:
            self.server.ledger.append(0.04)  # fake charge, not a real payment
            attempt = len(self.server.ledger)
        broken = self.server.mode == "disconnect" and attempt == 1
        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Connection", "close")
        if broken:
            self.send_header("Transfer-Encoding", "chunked")
        self.end_headers()

        def send(obj):
            data = ("data: " + json.dumps(obj) + "\n\n").encode()
            self.wfile.write(f"{len(data):x}\r\n".encode() + data + b"\r\n" if broken else data)
            self.wfile.flush()

        base = {"id": "fake", "object": "chat.completion.chunk", "created": 1, "model": "mock-model"}
        send({**base, "choices": [{"index": 0, "delta": {"role": "assistant", "content": "partial" if broken else "OK"}, "finish_reason": None}]})
        if broken:
            # Interrupt the HTTP chunked stream without the final chunk/usage.
            self.connection.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack("hh", 1, 0))
            self.close_connection = True
            return
        send({**base, "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]})
        send({**base, "choices": [], "usage": {"prompt_tokens": 10000, "completion_tokens": 2000, "total_tokens": 12000, "cost": 0.04}})
        self.wfile.write(b"data: [DONE]\n\n")
        self.wfile.flush()
        self.close_connection = True


class MockServer(ThreadingHTTPServer):
    def shutdown_request(self, request):
        request.close()  # do not turn the intended RST into a graceful shutdown


def scenario(mode):
    root = Path(tempfile.mkdtemp(prefix="opencode-retry-mock-"))
    for name in ("home", "config", "data", "cache", "state", "project"):
        (root / name).mkdir()
    server = MockServer(("127.0.0.1", 0), MockHandler)
    server.mode, server.ledger, server.lock = mode, [], threading.Lock()
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()
    config = {
        "$schema": "https://opencode.ai/config.json", "model": "mock/mock-model", "update": "disable", "snapshots": False,
        "policies": [{"action": "provider.use", "resource": "*", "effect": "deny"}, {"action": "provider.use", "resource": "mock", "effect": "allow"}],
        "providers": {"mock": {"name": "Local mock", "package": "@opencode/ai/providers/openai-compatible", "settings": {"baseURL": f"http://127.0.0.1:{server.server_address[1]}/v1", "apiKey": "fake-local-key"}, "models": {"mock-model": {"name": "Fake model", "limit": {"context": 100000, "output": 4000}}}}},
    }
    (root / "project" / "opencode.json").write_text(json.dumps(config), encoding="utf-8")
    keep = {"SYSTEMROOT", "WINDIR", "COMSPEC", "PATH", "PATHEXT", "TEMP", "TMP", "PROCESSOR_ARCHITECTURE", "NUMBER_OF_PROCESSORS"}
    env = {k: v for k, v in os.environ.items() if k.upper() in keep}
    env.update({"HOME": str(root / "home"), "USERPROFILE": str(root / "home"), "XDG_CONFIG_HOME": str(root / "config"), "XDG_DATA_HOME": str(root / "data"), "XDG_CACHE_HOME": str(root / "cache"), "XDG_STATE_HOME": str(root / "state"), "OPENCODE_DB": str(root / "test.db"), "NO_PROXY": "127.0.0.1,localhost", "OPENCODE_DISABLE_FREE_MODELS": "1"})
    cmd = [CLI, "--standalone", "run", "--model", "mock/mock-model", "--title", "Mock test", "--format", "json", "Say OK"]
    try:
        proc = subprocess.run(cmd, cwd=root / "project", env=env, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=90)
    finally:
        server.shutdown()
        server.server_close()
        thread.join(timeout=5)
    if proc.returncode:
        raise RuntimeError(f"OpenCode failed: {proc.stderr}\n{proc.stdout}")
    db = sqlite3.connect(f"file:{(root / 'test.db').as_posix()}?mode=ro", uri=True)
    cost = db.execute("SELECT cost FROM session_v2").fetchone()[0]
    steps = [json.loads(row[0]) for row in db.execute("SELECT data FROM session_message WHERE type='assistant' ORDER BY seq")]
    db.close()
    events = [json.loads(line) for line in proc.stdout.splitlines() if line.startswith("{")]
    retries = [e.get("error", {}).get("type") for e in events if e.get("type") == "retry"]
    print(json.dumps({"mode": mode, "mock_attempts": len(server.ledger), "mock_total": round(sum(server.ledger), 2), "session_total": cost, "assistant_steps": len(steps), "retry_errors": retries}))
    if mode == "control":
        assert len(server.ledger) == 1 and abs(cost - 0.04) < 1e-8
    else:
        assert len(server.ledger) == 2 and abs(cost - 0.04) < 1e-8 and "provider.transport" in retries


scenario("control")
scenario("disconnect")

Lingua principale
TypeScript
Stelle
227k
Fork
30.7k
Merge medio
3h 57m
PR unite (30g)
484

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di anomalyco/opencode

Tutte le issue di anomalyco/opencode

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.