Tool-calling runs get quadratically slower: completed tool calls are re-fingerprinted every turn
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 32/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 技术栈
- python
- 领域
- backend, performance
调研方向
Start in src/agents/run_internal/tool_planning.py at _dedupe_processed_response_invocations, and run the issue's fake-model benchmark at 50, 100 and 300 turns to confirm the growth. The fix has to keep duplicate-call-ID errors and outputs identical in both run and stream, so check the existing tests for those paths before editing. Done means the per-turn cost stops growing with history length and the outputs match the current behaviour.
由索引模型根据 Issue 内容生成。
描述
Describe the bug
Run time grows quadratically with the number of tool-calling turns. On every turn, _dedupe_processed_response_invocations (src/agents/run_internal/tool_planning.py) recomputes the canonical identity of every completed tool call in the run history: model_dump, argument normalisation and two SHA-256 digests per call. It also model_dumps every history item just to read its type. With one tool call per turn, the per-turn cost is O(history), so a run costs O(turns²).
With a fake model, one tool call per turn, a 300-turn run takes 2.69 s on main (17d994b). Of that, about 1.2 s is this re-fingerprinting.
Root cause and a possible fix
The history fingerprints are only needed for call IDs that can collide: IDs that are reused within the history, or that appear in the current response. Fingerprinting only those call IDs, skipping model_dump when the model's type already rules the item out, and not rebuilding ToolApprovalItems when legacy reconstruction is off keeps the duplicate-ID errors and all outputs unchanged. new_items and to_input_list() hash identically in both run and stream.
I have this change with a regression test on a fork branch, in case it is useful to a collaborator: https://github.com/deadstrobe5/openai-agents-python/compare/main...avoid-history-fingerprinting-per-turn
Fake model, one tool call per turn, CPython 3.14, 5 interleaved runs per side, median (range):
| turns | main | with the change |
|---|---|---|
| 50 | 141 ms (138-178) | 104 ms (102-106) |
| 100 | 385 ms (379-415) | 244 ms (240-261) |
| 200 streamed | 1300 ms (1293-1328) | 766 ms (747-788) |
| 300 | 2691 ms (2625-2796) | 1481 ms (1436-1505) |
Debug information
- Agents SDK version: 0.24.0 (
mainat 17d994b) - Python version: 3.14
- Operating system: macOS
- Model and model provider: fake
Model(no network) - Does the issue reproduce with the latest Agents SDK release? Yes
- Does the issue occur consistently or intermittently? Consistently
Repro steps
import asyncio, sys, time, json
from agents import Agent, Runner, function_tool, ModelResponse, Usage, RunConfig
from agents.models.interface import Model
from openai.types.responses import ResponseFunctionToolCall, ResponseOutputMessage, ResponseOutputText
N = int(sys.argv[1]) if len(sys.argv) > 1 else 200
MODE = sys.argv[2] if len(sys.argv) > 2 else "run"
@function_tool
def lookup(q: str) -> str:
return "result for " + q + " " + "x" * 200
class FakeModel(Model):
def __init__(self, n):
self.i = 0; self.n = n
def _out(self):
self.i += 1
if self.i > self.n:
return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
content=[ResponseOutputText(text="done", type="output_text", annotations=[], logprobs=[])])]
return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
content=[ResponseOutputText(text="thinking " * 20, type="output_text", annotations=[], logprobs=[])]),
ResponseFunctionToolCall(id=f"f{self.i}", call_id=f"c{self.i}", type="function_call", name="lookup",
arguments=json.dumps({"q": f"query {self.i}"}))]
async def get_response(self, *a, **k):
return ModelResponse(output=self._out(), usage=Usage(), response_id=None)
async def stream_response(self, *a, **k):
from openai.types.responses import ResponseCompletedEvent, Response
out = self._out()
yield ResponseCompletedEvent(type="response.completed", sequence_number=0, response=Response(
id="r", created_at=0, model="fake", object="response", output=out, tool_choice="auto", tools=[],
parallel_tool_calls=True))
async def main():
agent = Agent(name="a", instructions="x", model=FakeModel(N), tools=[lookup])
t = time.perf_counter()
if MODE == "stream":
r = Runner.run_streamed(agent, "hi", max_turns=N + 5)
async for _ in r.stream_events(): pass
else:
r = await Runner.run(agent, "hi", max_turns=N + 5)
dt = time.perf_counter() - t
print(f"{MODE} N={N} {dt*1000:.1f} ms items={len(r.new_items)} out={r.final_output!r}")
return r
if __name__ == "__main__":
asyncio.run(main())
python bench.py 300 or python bench.py 200 stream. Doubling the turns roughly quadruples the time spent in _dedupe_processed_response_invocations.
Expected behavior
Per-turn overhead that does not grow with the length of the run.
- 主要语言
- Python
- 星标
- 30.4k
- 派生
- 4.9k
- 平均合并
- 19 小时 5 分钟
- 30 天内合并 PR
- 90
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
openai/openai-agents-python 的其他 Issue
-
Docs: `tool_choice="required"` can select a state-changing tool when the needed one is not shown未关闭
难度 1/5 1 小时以内 新手友好度 90/100
openai/openai-agents-python#5461 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
openai/openai-agents-python#5460 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 85/100
openai/openai-agents-python#5459 · 4 条评论 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 90/100
openai/openai-agents-python#5458 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 85/100
openai/openai-agents-python#5455 ·
维护者通常 1 天内回复
查看 openai/openai-agents-python 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 72/100
pytest-dev/pyfakefs#1367 · 1 条评论 ·
维护者通常 2 天内回复
-
IFEvalG exclude_word_harder misses the keyword next to punctuation, at the start or end of the text, and in other letter case可能已有人在做 @damienen 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
allenai/open-instruct#2151 ·
维护者通常 1 天内回复
-
feat(plan): support convertorLabels and convertorAffinity in Plan resource可能已有人在做 @AmenB 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
RedHatQE/openshift-python-wrapper#2849 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
维护者通常 25 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
libnativeapi/nativeapi#21 ·