Parent: LLM Vulnerability Scanner Feature Parity with Garak, Giskard, CyberSecEval
维护者通常 2 天内回复
@abdelhadi703 已经在做这个了。
开始于 2026年10月6日。
评估
调研方向
未确定实现文件、测试或入口点。首先比较 Garak、Giskard 和 CyberSecEval 支持的攻击场景与 PyRIT 已记录的功能，然后针对一个缺失的场景创建一个聚焦的后续 issue；当该场景具有明确的实现或文档范围时，即视为完成。
由索引模型根据 Issue 内容生成。
描述
TLDR; We want help creating issues where Garak, Giskard, or CyberSecEval support an attack scenario where PyRIT doesn't support it.
Background
Other tools like Garak, Giskard, and CyberSecEval have different use cases from PyRIT, but there is overlap as well. If there is an attack in one of these other platforms, we probably want to support it also and would love issues being created so we can track this (or clarify if it already exists/improve our docs).
This paper tackles a really difficult problem, and that's comparing these tools (and PyRIT). When I was looking, some of the things I noticed that it mentioned PyRIT was lacking were:
- Lack of multi-language support (I think we actually have this as a converter)
- Lack of GCG (we have this as an auxiliary module)
- Dan, AutoDan, etc (I think we have support for all of this. We certainly have DAN and ways to modify those prompts with GPTFuzz - but we would love to know what we're missing)
- Insecure Coding Test (we don't have this, but it would be an easy scorer to write!)
And there might be more that I missed!
So as an example of issues we'd like, all of the above are good to open. Even if we think we support them (like in the multi-language case) we can dive into specifics to make sure we're not missing anything, and potentially improve documentation so it's more obvious these scenarios are supported.
Describe the solution you'd like
Please open follow-up sub-issues with any features you notice or find in these other tools that PyRIT doesn't have! That way we can track them. Also, we would love community to tackle any of these also, but it still can be helpful to open an issue so we can help guide :D
- 主要语言
- Python
- 星标
- 4.6k
- 派生
- 924
- 平均合并
- 2 天 14 小时
- 30 天内合并 PR
- 227
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
microsoft/PyRIT 的其他 Issue
-
BUG: PlagiarismScorer accepts invalid n-gram size and blank reference text可能已有人在做 @RohithPariki 于 3 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 85/100
维护者通常 2 天内回复
-
PackageHallucinationScorer (Python) misses `from pkg.sub import x` and indented imports可能已有人在做 @barry166 于 4 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
microsoft/PyRIT#2948 · 1 条评论 ·
维护者通常 2 天内回复
-
LiteLLMChatTarget does not flag or survive output-token truncation可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 2 天内回复
-
BUG Configuration keeps runtime-status errors after polling recovers可能已有人在做 @rupayon123 于 10 天前认领。 未关闭Bug: triage GUI help wanted
难度 2/5 1-3 小时 新手友好度 86/100
microsoft/PyRIT#2868 · 1 条评论 ·
维护者通常 2 天内回复
-
ObjectiveScorerEvaluator scores every conversation message as an assistant response可能已有人在做 @feiiiiii5 于 11 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 2 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
area:docs
难度 2/5 1-3 小时 新手友好度 72/100
RailtownAI/railtracks#1633 ·
维护者通常 2 天内回复
-
review-panel severity:low
难度 1/5 1 小时以内 新手友好度 85/100
kristovatlas/coin-accounting#152 ·
维护者通常 1 天内回复
-
documentation :blue_book:
难度 1/5 1 小时以内 新手友好度 88/100
PennyLaneAI/pennylane#10280 ·
维护者通常 2 天内回复
-
`pipx reinstall` prints a Python traceback when the reinstall fails可能已有人在做 @ParamTanna 今天认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复