fix(supervisor): 提案阶段失败与「提案池为空」共用 no_proposal 终态——last_error 诊断不出结构,自改进循环静默收工
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 60/100
Research direction
Start in plaita-ai/plaita_ai/supervisor.py: Supervisor.iterate (around line 512) where a missing proposal sets reason, and the run_loop return dict (around line 623). Reproduce with the script in the issue, then read the tests at plaita-ai/tests/test_supervisor.py (lines 175-185) and tests/test_mfix_f_flow_source_proposer.py. Done when a failed proposal exposes non-empty proposer_error text in the result while an empty static pool stays empty and final_status remains no_proposal.
Written by the indexing model from the issue text.
Description
优先级 P2 · 依赖:无
基线 HEAD = 65f48f2
Summary
FlowSourceProposer 在编译门耗尽或模型输出不可解析时返回 None,Supervisor.iterate
把它折叠成与「提案池本来就是空的」完全相同的 status="no_proposal"(reason 恒为
"proposer returned no candidate");唯一能区分两种成因的诊断 proposer.last_error
/ last_source 只留在 proposer 对象上,run_loop 的返回结构里没有任何字段承载它。
证据 1 —— 诊断字段的自我声明:它只在 propose 返回 None 时非空,但没有任何出口:
plaita-ai/plaita_ai/supervisor.py:319-321
#: 编译门耗尽后的最后错误(诊断用;propose 返回 None 时非空)
self.last_error: Optional[str] = None
self.last_source: Optional[str] = None
证据 2 —— 两条失败路径都写 last_error 后 continue / 返回 None:
plaita-ai/plaita_ai/supervisor.py:424-444
try:
source = extract_flow_source(content)
except ValueError as exc:
# 输出里没有 @flow 源码:烧一次 attempt,把可操作错误回喂下一轮。
self.last_error = f"attempt {attempt}: {exc}"
errors_text = str(exc)
parse_feedback = str(exc)
continue
compiled = compile_flow(source, flow_id=flow_id or None)
if compiled.ok and compiled.ir is not None:
self.last_source = source
self.last_error = None
return {
"definition": json.dumps(compiled.ir, ensure_ascii=False, default=str),
"rationale": rationale
or f"compiled @flow source ({attempt} attempt(s), flow_id={flow_id})",
}
errors_text = format_compile_errors(compiled.errors)
self.last_error = f"attempt {attempt}: {errors_text}"
return None
证据 3 —— iterate 不读回 last_error,直接与静态空池共用同一终态与同一 reason:
plaita-ai/plaita_ai/supervisor.py:512-515
proposal = self.proposer.propose(context)
if not proposal or not proposal.get("definition"):
result["reason"] = "proposer returned no candidate"
return result
与之共用状态的是 StaticProposer.propose 的候选耗尽返回 None(supervisor.py:72-74):
def propose(self, context: Dict[str, Any]) -> Optional[Dict[str, Any]]:
if self._cursor >= len(self.candidates):
return None
证据 4 —— 返回结构里没有任何 error 字段,last_error 全仓无生产读点:
plaita-ai/plaita_ai/supervisor.py:623-628
return {
"flow_id": flow_id,
"final_status": final_status,
"iterations": iterations,
"iterations_run": len(iterations),
}
$ grep -rn "last_error" --include='*.py' plaita-ai/ | grep -v __pycache__
plaita-ai/tests/test_mfix_f_flow_source_proposer.py:149: assert proposer.last_error is None
plaita-ai/tests/test_mfix_f_flow_source_proposer.py:153: """回喂耗尽 → propose 返回 None、last_error 记录;Supervisor 按停,不写版本。"""
plaita-ai/tests/test_mfix_f_flow_source_proposer.py:164: assert proposer.last_error is not None and "f-string" in proposer.last_error
plaita-ai/tests/test_mfix_f_flow_source_proposer.py:183: assert "@flow" in str(proposer.last_error)
plaita-ai/plaita_ai/supervisor.py:247: 最后一次错误留在 ``self.last_error`` 供诊断。
plaita-ai/plaita_ai/supervisor.py:320: self.last_error: Optional[str] = None
plaita-ai/plaita_ai/supervisor.py:398: self.last_error = None
plaita-ai/plaita_ai/supervisor.py:428: self.last_error = f"attempt {attempt}: {exc}"
plaita-ai/plaita_ai/supervisor.py:435: self.last_error = None
plaita-ai/plaita_ai/supervisor.py:442: self.last_error = f"attempt {attempt}: {errors_text}"
即:写入点 4 处全在 supervisor.py,读取点只在 tests/——run_loop 的两条生产入口
(ops_mcp.py:222-257 的 supervisor_iterate、plaita-console/backend/api/supervisor.py:105-145
的面板 POST /flows/{id}/supervisor/iterate)都只是 return supervisor.run_loop(...),
拿不到该字段。
复现(实跑,输出逐字照抄):
$ cd /Users/jeff.jie/projects/infra4agent/plaita/plaita-ai && PYTHONPATH=. /tmp/plaita-audit-venv/bin/python - <<'EOF'
import json
from types import SimpleNamespace
from plaita_ai.console_client import ConsoleClient, ConsoleConfig
from plaita_ai.evals import Dataset
from plaita_ai.supervisor import Supervisor, SupervisorPolicy, FlowSourceProposer
from tests._console_fake import FakeConsole
fake = FakeConsole(); fake.seed_flow("demo", {"1.0.0": '{"v": 1}'}, published="1.0.0")
fake.dry_run_handler = lambda f, i: {"result": {"text": "good"}, "nodes": [], "error": None}
client = ConsoleClient(ConsoleConfig(base_url="http://fake", admin_api_key="test-key"), transport=fake.transport())
ds = Dataset([{"id": "c1", "input": {}, "expect": {"contains": "good"}}], source="ds")
class ProseModel:
def invoke(self, messages, *a, **k):
return SimpleNamespace(content="Sure, I would improve the flow by adding a retry node.")
proposer = FlowSourceProposer(model=ProseModel())
sup = Supervisor(client, SupervisorPolicy(max_iterations=5), proposer)
out = sup.run_loop("demo", ds)
print(json.dumps(out, ensure_ascii=False))
print("last_error =", proposer.last_error)
print("'error' in result:", "error" in json.dumps(out))
EOF
{"flow_id": "demo", "final_status": "no_proposal", "iterations": [{"flow_id": "demo", "baseline_version": "1.0.0", "baseline_avg_score": 1.0, "status": "no_proposal", "reason": "proposer returned no candidate", "iteration": 1}], "iterations_run": 1}
last_error = attempt 3: 未能从模型输出中解析 @flow 源码(需要只输出一个包含 @flow 的 ```python ... ``` 代码块,不要输出额外说明文字)
'error' in result: False
注意 attempt 3:三轮编译门全部耗尽,返回体里只有一句 "proposer returned no candidate",
看起来与 StaticProposer([]) 空池一模一样(后者由 tests/test_supervisor.py:175-185 钉住)。
影响
- 谁踩到:用默认
proposer="flow"跑自迭代的两条生产链路——MCP 工具supervisor_iterate
(ops_mcp.py:222-257,proposer默认"flow")与 console 面板
POST/api/flows/{id}/supervisor/iterate(api/supervisor.py:105-145,proposer默认flow)。 - 代价是不可观测而非数据损坏:当上游模型持续输出散文/半截代码、或产出的 @flow 一直编译不过
(典型的模型能力不足、DSL 参考被裁掉、选错模型名、base_url 打到非 chat 端点)时,
run_loop立刻返回final_status="no_proposal"、iterations_run=1并停表。调用方
与面板读到的是一个「没有提案可评」的正常终止,看不出是「模型坏了」还是「提案池空了」。 - 失败现场(
attempt 3: ...的编译器错误列表或解析错误,恰是重跑/换模型/改 prompt 的唯一线索)
随进程结束即丢;last_error只在还在调用栈里的proposer对象上,而调用方从不持有它。 - 触发不需要任何异常或边界条件:只要模型输出不含
python里的 @flow 源码就能复现,
即默认首跑就可能命中。不修的后果:自改进链路静默收工且无法归因,运维只能靠反复手工试模型。 - 附带的一致性缺口:
PromptProposer.propose在上游 HTTP 错误上直接
resp.raise_for_status()(supervisor.py:137)抛出,而_guarded
(ops_mcp.py:46-60)只兜ConsoleClientError/(ValueError, KeyError),httpx 异常会
穿透成工具级错误;同类上游故障下flow提案静默收工、prompt提案整体报错,两条路径行为相反。
建议
- 让
run_loop的结果能自证成因:在Supervisor.iterate(supervisor.py:512-515)
取不到提案时,回读getattr(self.proposer, "last_error", None)/last_source,把非空值写入
该 iteration 的独立字段(例如result["proposer_error"],与reason分开);run_loop
的返回体(supervisor.py:623-628)再带上final_status的成因摘要(一个error字段即可)。
区分语义建议:池空 →no_proposal;有诊断 → 同一终态但reason/error能读出
「propose 失败」。 FlowSourceProposer.propose返回None前可保留last_error的最终值(当前在成功分支
:435会置回None,语义正确;只需确保失败分支不因任何路径丢值)。- 统一两条 proposer 的上游失败行为:要么
PromptProposer.propose(supervisor.py:137)
仿FlowSourceProposer收敛成带诊断的None,要么在_guarded(ops_mcp.py:46-60)
增补httpx.HTTPError的兜底并给出ok:false载荷——二选一,不要停在现在的相反行为上。
验收条件(可执行、可断言):
- 新增测试(可放
plaita-ai/tests/test_supervisor.py):用返回散文的 duck-type 模型构造
FlowSourceProposer(max_compile_attempts=3),跑
Supervisor(client, SupervisorPolicy(max_iterations=5), proposer).run_loop("demo", ds),
断言返回体(或其iterations[0])中存在承载proposer.last_error的非空字段,
且该字段包含"attempt 3"与解析/编译错误文本;同时断言final_status仍为"no_proposal"
(不改变既有终态语义,tests/test_supervisor.py:175-185继续绿)。 - 反向断言「真池空」与「提案失败」不再同形:
StaticProposer([])的返回体该项为空/缺省,
只有失败路径才非空。 - 若选择统一 proposer 行为,补一条
_guarded用例:工具体内抛httpx.HTTPStatusError
时返回{"ok": false, ...}而不是让异常穿透。
边界
- 不重设计 proposer 契约、不改
no_proposal这一终态本身,也不改「编译门耗尽不写版本号、
不烧迭代」的既定语义(supervisor.py:246-247、supervisor_flow.py:90的同名降级路径保持原样)。 - 不顺手重构
ops_mcp.py的_guarded覆盖面(第 3 点只是指出两条 proposer 行为相反,
实现归本单的验收项之一;不扩大为通用异常兜底改造)。 - 与
evaluate_proposal段的候选版本号/乐观锁问题无关(那是另一处代码:supervisor.py:538-545
与console_client.py:152-164),本单不触碰。 - 查重:
gh issue list -R jeffkit/plaita --state all --limit 100 --json number,title共 33 条,
标题中无 supervisor/proposer/no_proposal/自迭代相关条目,未发现同根既有单。
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- Avg merge
- 2h 22m
- Merged PRs (30d)
- 8
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from jeffkit/plaita
-
fix(console): #274 并发同 dedup_key 双 POST 各自入队——预认领与占位行非原子,落败 id 永久 queued 且占配额Possibly taken @okguitar claimed this today. Open
Difficulty 5/5 Over a week Newbie friendliness 35/100
jeffkit/plaita#280 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 1 day
-
fix(console): ${redis.url} 从配置文件而非数据面 effective URL 解析——metadata 档 worker 静默连 localhost,warnings 为空Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Maintainers usually reply within 1 day
-
refactor(expr): 摘除运行期字符串的二次求值(#104/#130 的终章)——契约定为「配置求值、运行期数据」Possibly taken @jeffkit claimed this 1 day ago. Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
jeffkit/plaita#268 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
developmentseed/titiler-cmr#250 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug]: IndexHandler.post crashes with HTTP 500 on malformed payloads, missing credentials, or when login is disabledPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 6 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
cookiecutter/cookiecutter-django#6999 ·
Maintainers usually reply within 1 day
-
fix: @_helpers.positional raises TypeError when decorating functions without default argumentsPossibly taken @codeXsidd claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
googleapis/google-api-python-client#2845 ·
Maintainers usually reply within 1 day
-
github_actions
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
collective/icalendar#2003 · 1 comment ·
Maintainers usually reply within 1 day