Tool-calling runs get quadratically slower: completed tool calls are re-fingerprinted every turn

已关闭
#5,382 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
32/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python

调研方向

Start in src/agents/run_internal/tool_planning.py at _dedupe_processed_response_invocations, and run the issue's fake-model benchmark at 50, 100 and 300 turns to confirm the growth. The fix has to keep duplicate-call-ID errors and outputs identical in both run and stream, so check the existing tests for those paths before editing. Done means the per-turn cost stops growing with history length and the outputs match the current behaviour.

由索引模型根据 Issue 内容生成。

描述

Describe the bug

Run time grows quadratically with the number of tool-calling turns. On every turn, _dedupe_processed_response_invocations (src/agents/run_internal/tool_planning.py) recomputes the canonical identity of every completed tool call in the run history: model_dump, argument normalisation and two SHA-256 digests per call. It also model_dumps every history item just to read its type. With one tool call per turn, the per-turn cost is O(history), so a run costs O(turns²).

With a fake model, one tool call per turn, a 300-turn run takes 2.69 s on main (17d994b). Of that, about 1.2 s is this re-fingerprinting.

Root cause and a possible fix

The history fingerprints are only needed for call IDs that can collide: IDs that are reused within the history, or that appear in the current response. Fingerprinting only those call IDs, skipping model_dump when the model's type already rules the item out, and not rebuilding ToolApprovalItems when legacy reconstruction is off keeps the duplicate-ID errors and all outputs unchanged. new_items and to_input_list() hash identically in both run and stream.

I have this change with a regression test on a fork branch, in case it is useful to a collaborator: https://github.com/deadstrobe5/openai-agents-python/compare/main...avoid-history-fingerprinting-per-turn

Fake model, one tool call per turn, CPython 3.14, 5 interleaved runs per side, median (range):

turns main with the change
50 141 ms (138-178) 104 ms (102-106)
100 385 ms (379-415) 244 ms (240-261)
200 streamed 1300 ms (1293-1328) 766 ms (747-788)
300 2691 ms (2625-2796) 1481 ms (1436-1505)
Debug information
  • Agents SDK version: 0.24.0 (main at 17d994b)
  • Python version: 3.14
  • Operating system: macOS
  • Model and model provider: fake Model (no network)
  • Does the issue reproduce with the latest Agents SDK release? Yes
  • Does the issue occur consistently or intermittently? Consistently
Repro steps
import asyncio, sys, time, json
from agents import Agent, Runner, function_tool, ModelResponse, Usage, RunConfig
from agents.models.interface import Model
from openai.types.responses import ResponseFunctionToolCall, ResponseOutputMessage, ResponseOutputText

N = int(sys.argv[1]) if len(sys.argv) > 1 else 200
MODE = sys.argv[2] if len(sys.argv) > 2 else "run"

@function_tool
def lookup(q: str) -> str:
    return "result for " + q + " " + "x" * 200

class FakeModel(Model):
    def __init__(self, n):
        self.i = 0; self.n = n
    def _out(self):
        self.i += 1
        if self.i > self.n:
            return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
                content=[ResponseOutputText(text="done", type="output_text", annotations=[], logprobs=[])])]
        return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
                    content=[ResponseOutputText(text="thinking " * 20, type="output_text", annotations=[], logprobs=[])]),
                ResponseFunctionToolCall(id=f"f{self.i}", call_id=f"c{self.i}", type="function_call", name="lookup",
                    arguments=json.dumps({"q": f"query {self.i}"}))]
    async def get_response(self, *a, **k):
        return ModelResponse(output=self._out(), usage=Usage(), response_id=None)
    async def stream_response(self, *a, **k):
        from openai.types.responses import ResponseCompletedEvent, Response
        out = self._out()
        yield ResponseCompletedEvent(type="response.completed", sequence_number=0, response=Response(
            id="r", created_at=0, model="fake", object="response", output=out, tool_choice="auto", tools=[],
            parallel_tool_calls=True))

async def main():
    agent = Agent(name="a", instructions="x", model=FakeModel(N), tools=[lookup])
    t = time.perf_counter()
    if MODE == "stream":
        r = Runner.run_streamed(agent, "hi", max_turns=N + 5)
        async for _ in r.stream_events(): pass
    else:
        r = await Runner.run(agent, "hi", max_turns=N + 5)
    dt = time.perf_counter() - t
    print(f"{MODE} N={N} {dt*1000:.1f} ms items={len(r.new_items)} out={r.final_output!r}")
    return r

if __name__ == "__main__":
    asyncio.run(main())

python bench.py 300 or python bench.py 200 stream. Doubling the turns roughly quadruples the time spent in _dedupe_processed_response_invocations.

Expected behavior

Per-turn overhead that does not grow with the length of the run.

主要语言
Python
星标
30.4k
派生
4.9k
平均合并
19 小时 5 分钟
30 天内合并 PR
90

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

openai/openai-agents-python 的其他 Issue

查看 openai/openai-agents-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。