Title: Feature Request: Attack strategy for Excessive Agency / unauthorized tool invocation
维护者通常 2 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start by reading openai_response_target.py and the existing Tool/ToolProvider interfaces, then compare the single-turn and multi-turn attack structures, including Crescendo. Confirm with maintainers how tool-call output can be inspected and scored. Done means an agreed attack design and implementation that evaluates actual unauthorized or excessive tool calls rather than only text output.
由索引模型根据 Issue 内容生成。
描述
Summary
PyRIT already supports exposing tools to a target via Tool / ToolProvider
(see openai_response_target.py), but I don't see an attack strategy that
specifically tests Excessive Agency — whether an agent can be manipulated
into invoking a tool outside its intended scope, chaining tool calls beyond
what a task requires, or calling a tool that its system prompt explicitly
restricts.
This maps to the OWASP Top 10 for LLM Applications (Excessive Agency /
Insecure Plugin Design) and is a distinct risk category from prompt injection
or jailbreaking the model's text output — it targets the agent's actions,
not just its words.
Proposed approach (open to feedback before implementing)
- A new attack, e.g.
ExcessiveAgencyAttack, that:- Takes a target configured with a defined set of "allowed" tools/scope
(via the existingTool/ToolProvidersystem). - Attempts to elicit a tool call outside that scope — either a tool the
agent has access to but shouldn't use for the stated task, or a chained
sequence of legitimate calls that together exceed the intended
permission boundary. - Scores success based on the tool call actually made (inspecting the
target's tool-call output), not on the text response — this is the
distinct piece existing text/output scorers don't cover.
- Takes a target configured with a defined set of "allowed" tools/scope
- Could reuse the single-turn or multi-turn structure depending on whether
the elicitation needs conversation history (multi-turn is likely more
realistic here, similar to how Crescendo works for text escalation).
Questions for maintainers
- Is there existing tooling for asserting on tool-call output specifically
(as opposed to text output) that I should reuse for scoring? - Any prior discussion/PR on agentic risk testing I should be aware of
before designing this?
Happy to take this on once the approach is validated.
- 主要语言
- Python
- 星标
- 4.5k
- 派生
- 896
- 平均合并
- 2 天 22 小时
- 30 天内合并 PR
- 214
环境准备
我们还没有检查这个项目的环境配置文件。先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
microsoft/PyRIT 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 88/100
microsoft/PyRIT#2905 · 3 条评论 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 2 天内回复
-
Bug: triage GUI help wanted
难度 2/5 1-3 小时 新手友好度 86/100
microsoft/PyRIT#2868 · 1 条评论 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 2 天内回复
-
难度 3/5 1-2 天 新手友好度 72/100
维护者通常 2 天内回复
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 85/100
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 90/100
维护者通常 1 天内回复
-
instance instance add
难度 2/5 1-3 小时 新手友好度 68/100
searxng/searx-instances#941 · 1 条评论 ·
-
难度 1/5 1 小时以内 新手友好度 92/100
FluidNumerics/fluid-walk-blocker#89 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复