Retry layer misclassifies deterministic 400s as transient, workflow steps replay non-idempotent work, and AgentTeam.wait_for_completions() loses finished sub-agents

オープン
#5,755 コメント 5 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
30/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
backend

調査の方向性

This bundles three separate bugs. Start with classify_error_kind in llm/llm.py (around lines 977-1017) and the retry loops _call_with_retry and _call_with_retry_async, then Workflow step retry in workflows/workflows.py (around 3360) and wait_for_completions / _handle_sub_agent_completion in agents/agents.py (around 4461). Done means a 400 or programming error fails on the first attempt, a non-transient function step runs once, and wait_for_completions() returns already-finished sub-agents with a bounded _completion_events. Split it into separate PRs and agree the retry defaults with maintainers first.

索引モデルが issue の本文から書いたものです。

説明

bug claude documentation origin/outsider

Summary

An analysis of src/praisonai-agents/praisonaiagents found three validated gaps in how failures and completions are handled. They follow the same pattern: failures and completions are inferred from loose signals (substrings, "any exception", deleted records) instead of the real signal. All three were confirmed against main at f8bf0921. Findings 1 and 2 have a reproduction below. Finding 3 is confirmed by code reading and the agent's own reproduction.

Scope: core SDK only. Docs, tests and file size are out of scope.


1. classify_error_kind matches bare digits in messages, so deterministic 400s are retried as "overloaded" or "rate_limit"

Where: llm/llm.py:977-979 (rate_limit) and llm/llm.py:1015-1017 (overloaded)

# rate limit
if any(indicator in error_str for indicator in [
    "rate limit", "ratelimit", "too many request", "resource_exhausted",
    "usage limit", "429"                      # <- matches "6429", "max 4290", ...
]) or "429" in str(getattr(error, "status_code", "")):
    return "rate_limit"
...
# overloaded
"503", "502", "500"                           # <- matches "max_tokens=5000"

status_code is only consulted for 429. A 400 whose message contains "500", "502", "503" or "429" is treated as transient. _call_with_retry and _call_with_retry_async then retry it with backoff (llm/llm.py:~1473 and ~1559).

Reproduction (stubbed litellm, no network):

class E(Exception): status_code = 400
o = LLM.__new__(LLM)
o.classify_error_kind(E("Invalid value: max_tokens=5000 exceeds limit 4096"))  # -> 'overloaded'
o.classify_error_kind(E("user id 6429 not permitted"))                          # -> 'rate_limit'

Observed with a fake client:

Error (status 400) kind calls made sleeps
max_tokens=5000 exceeds limit 4096 overloaded 4 2s, 4s, 8s
n must be <= 128, got 1500 overloaded 4 2s, 4s, 8s
user id 6429 not permitted rate_limit 4 60s, 60s, 60s (180s total)
plain BadRequestError / ContentPolicyViolationError unknown 3 1s, 2s
ValueError / TypeError / KeyError unknown 3 1s, 2s

A client error that can never succeed ties up an agent for up to 3 minutes and bills 4 requests.

Fix: use status_code (or the exception class) first, and only fall back to message matching when no status is available.

status = getattr(error, "status_code", None) or getattr(getattr(error, "response", None), "status_code", None)
if isinstance(status, int):
    if status == 429: return "rate_limit"
    if status in (500, 502, 503, 504, 529): return "overloaded"
    if status in (400, 404, 413, 422): return "non_retryable"   # new kind: fail fast
# ... existing substring heuristics only when status is None,
# using word boundaries: re.search(r"\b429\b", error_str)

Programming errors (ValueError, TypeError, KeyError) should not enter the retry loop.


2. Workflow steps retry every exception 3 times by default, including non-idempotent function steps

Where: workflows/workflows.py:3360-3380, with the same pattern near :1930-1936 and :2113

max_retries = getattr(step, 'max_retries', 3)
...
except Exception as e:
    step_error = e
    is_retryable = getattr(e, 'is_retryable', True)   # <- default True for everything
    if not is_retryable:
        break
    retry_count += 1
    if retry_count <= max_retries:
        self._retry_backoff(...)
        continue

Any exception raised by user code, such as a ValueError from a bug, is replayed 3 more times with backoff. A function step that sends an email, charges a card or inserts a row runs 4 times.

Reproduction: Workflow(steps=[bad]) where bad raises ValueError:

  • 4 calls
  • 7.0s elapsed
  • retries: 4

The same 4x replay happens inside parallel([...]) branches.

Fix: make retries opt-in for deterministic failures and function steps, and retry only transient errors for agent steps.

_TRANSIENT = (TimeoutError, ConnectionError)   # plus LLM errors classified as rate_limit/overloaded (see #1)

def _should_retry(step, e) -> bool:
    explicit = getattr(e, 'is_retryable', None)
    if explicit is not None:
        return explicit
    if getattr(step, 'retry_on_error', False):   # opt-in for arbitrary exceptions
        return True
    return isinstance(e, _TRANSIENT)

Default max_retries for plain callables should be 0.


3. AgentTeam.wait_for_completions() returns nothing for fast sub-agents, and _completion_events grows without bound

Where: agents/agents.py:4461 (async variant :4678), cleanup at :4515-4519, append at :4424 (async :4635)

# wait_for_completions
with self._spawn_lock:
    target_agents = agent_ids or list(self._spawned_agents.keys())   # finished agents are already gone
...
# _handle_sub_agent_completion
if agent_id in self._spawned_agents:
    del self._spawned_agents[agent_id]                                # deleted on completion
...
self._completion_events.append(completion_event)                      # plain list, never trimmed

Completion removes the spawn record, so the default target set (agent_ids omitted) only contains agents still running. If every sub-agent already finished, target_agents is empty and the wait returns [] immediately. The async variant derives its targets the same way.

Reproduction: spawn 3 instant sub-agents and sleep 1s.

  • len(_spawned_agents) is 0 and len(_completion_events) is 3.
  • wait_for_completions(timeout=2) returns 0 results (expected 3).
  • With explicit agent_ids it returns 3.
  • After 2000 announce_completion(..., 'R'*1000) calls, len(_completion_events) is 2003, with each full result string retained.

Fix: derive default targets from both live spawns and recorded events, and bound the event store.

with self._spawn_lock:
    target_agents = agent_ids or (
        set(self._spawned_agents) | {e.agent_id for e in self._completion_events}
    )

# in __init__:
self._completion_events = collections.deque(maxlen=self._max_completion_events)   # e.g. 256

Alternatively, drain events once they are returned from wait_for_completions.


Also found, lower priority

  • memory/file_memory.py:544-567: _auto_promote_to_long_term bypasses long_term_limit, rebuilds existing_ids per item, and rewrites the whole JSON file on every add. 1500 high-importance adds produced 1500 long-term entries (limit 1000) and took 23.6s.
  • llm/rate_limiter.py:167-218: acquire_tokens(n) with n above the bucket capacity never succeeds in the sync path, and silently goes negative in the async path. Nothing in the SDK calls it today.
  • memory/mcp_config.py:289,348, memory/hooks.py:417, memory/rules_manager.py:824, memory/docs_manager.py:390 use non-atomic write_text, unlike file_memory, session/store and storage/base, which use temp file plus os.replace.

Acceptance criteria

  • A 400, 404 or 422 response, and programming errors such as TypeError, fail on the first attempt with no retry sleeps.
  • A workflow function step that raises a non-transient exception runs once by default.
  • wait_for_completions() with no arguments returns results for sub-agents that completed before the call.
  • _completion_events has a bounded size.
主要言語
Python
スター
9.3k
フォーク
1.5k
平均マージ
1日 5時間
マージ済み PR(30日)
520

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

MervinPraison/PraisonAI のほかの issue

MervinPraison/PraisonAI の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。