fix(screener): 拦截「三态混同」+ flow 的置信度闸被 passthrough 关掉——低置信猜测与基础设施故障都以「判定为注入风险」永久 blocked
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 24/100
Research direction
Start in issue_keeper/screener.py: the Verdict is assembled where the flow result is read (~lines 485-491, where out["low_confidence"] is dropped) and where exceptions fall through (~328-330, 538-541). Then read how keeper.py:466-473 turns any unsafe verdict into a permanent blocked plus a public comment, and compare the flow cache ~/.issue-keeper/screener-flow.json against the validation proposed in point ⑤. Done means low-confidence verdicts and backend errors no longer yield blocked or accusatory comments, with tests covering both paths.
Written by the indexing model from the issue text.
Description
场景 gap 单（okguitar 提报）· 优先级 P0 · 依赖：无
基线 HEAD = c08c667。（与 OPEN #3 同族：#3 记录现象层，本单给出机制层根因与修复面。）
Summary（三方佐证：代码 + 运行缓存 + 已发出评论）
- 置信度闸被关：生产判定配置来自 console flow 且原样透传（
issue_keeper/screener.py:416-441取min_confidence与on_low_confidence）；运行缓存~/.issue-keeper/screener-flow.json（v1.0.4）实测min_confidence=0.8但on_low_confidence="passthrough"（plaita-nodes decision.py:388：passthrough = 低置信选择原样返回）；而screener.py:485-491只取out["choice"]/["confidence"]，丢弃out["low_confidence"]——阈值形同虚设，0.6-0.78 的猜测与 0.93 的判定同一文案输出。 - 基础设施失败 = 真判定：decision 异常/非 JSON/flow 异常统一走
Verdict(safe=False, reason=f"flow 后端: {exc}")（screener.py:328-330,484,538-541）→ keeper 统一 blocked + 公开评论（keeper.py:466-473+_public_reason空原因兜底文案「疑似指令注入或越权诱导」）。 - 实盘：10-03/04 keeper.log 共 23 条拦截评论——10 条 reason 是 screener 自身故障（
choice '' 不在决策空间×6、LLM 端点不完整×2、未返回 JSON×2，含模板占位$your_answer被回显）；13 条「判定为注入风险」中 9 条置信度 <0.8（0.60-0.78）。这些评论已公开贴在 recursive#97/100/102/107、plaita#30/37、agentproc#9/19 上；state.json 现存 10 条 blocked，唯一出口是人工 reopen。
影响
安全加固类技术 issue 被批量永久拦下；对公众输出「你在注入」的不实指控；运维无法区分「模型说 unsafe」与「screener 自己坏了」；每次 reopen 都是重新掷骰子（#97/#100/#107 各被拦 2 次）。
建议
① Verdict 增 error: bool / low_confidence: float|None；② low_confidence=True 一律按「不确定」处理（禁止进 blocked，转人工/自动重试），不受 flow 配置影响；③ 基础设施失败 = 自动重试 N 次 + 计数升级，不设 blocked；④ 公开文案区分「判定为注入（置信度 X）」与「安全过滤服务故障，待重试」；⑤ flow 定义校验（choices 非空、占位符残留、on_low_confidence 白名单）。
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from jeffkit/issue-keeper
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
jeffkit/issue-keeper#22 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
jeffkit/issue-keeper#21 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
jeffkit/issue-keeper#19 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
jeffkit/issue-keeper#17 · 6 comments ·
-
keeper-ignore
Difficulty 5/5 Over a week Newbie friendliness 20/100
jeffkit/issue-keeper#16 · 2 comments ·
All issues in jeffkit/issue-keeper
Similar issues
-
docs pydanty:is-working
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pydantic/pydantic-ai#10113 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
GreedyBear-Project/GreedyBear#1727 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
ARM-software/CMSIS-DSP#354 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
kiat/codeAssist#391 ·
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
darnitdevorg/darnit#587 ·
Maintainers usually reply within 1 day