FEAT: Support custom scenarios over selected dataset examples
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 35/100
調査の方向性
Keep this work blocked until prerequisites 12–13 are promoted. Start with Scenario, DatasetAttackConfiguration, MatrixAtomicAttackBuilder, AttackTechniqueRegistry, ScorerRegistry, and existing matrix-shaped scenarios; read doc\code\framework.md and the scenario/setup-technique/model/test instructions. Done means focused local-fixture tests cover exact selected populations, validation, shared estimates and launch configuration, provenance, progress, results/history, and resume behavior without changing existing scenarios.
索引モデルが issue の本文から書いたものです。
説明
Is your feature request related to a problem? Please describe.
The explorer's multi-select flow needs an ordinary scenario over exactly the selected examples, defaulting to prompt sending. Reusing another scenario's defaults could unexpectedly add techniques, sample away selections, or run prompt sending twice as both a technique and baseline.
Work item 14 of 17 in #2744. Technical prerequisites: items 12-13. Keep not ready yet until promoted.
Describe the solution you'd like
Add the smallest registered scenario/configuration adapter needed for explicit selected examples. Reuse Scenario, existing attack techniques/factories, atomic attack construction, scorer registry resolution, and normal scenario progress/results/history.
- The default effective attack is exactly prompt sending with no additional converters. Do not fall back to another scenario's default technique set or emit a duplicate baseline for the same population.
- Accept a compatible configured objective-scorer preset, defaulting to the compatible configured default. Require valid objectives; preserve source objectives and accept explicit user-supplied objectives for examples that need them instead of inventing goals.
- Resolve explicit selections through the contract from item 13. Retain group boundaries, roles, multimodal content, and provenance. Explain unsupported seed shapes rather than flattening templates or simulated configuration into text.
- Expose supported registered techniques and converter-composition seams for the later UI. Attacks/converters/scorers keep their normal responsibilities; the scenario packages them and owns execution orchestration.
- Use registry-compatible construction and metadata introspection without fetching datasets or generating conversations. Validate execution prerequisites at the appropriate run/setup boundary.
Acceptance criteria:
- N valid selected examples with default settings produce exactly N prompt-sending executions, not N plus an extra baseline/default attack set.
- Missing objectives/scorers, incompatible modalities/techniques, and invalid groups produce actionable validation errors.
- Read-only estimates and confirmed launch use the same explicit selection and effective configuration.
- Normal scenario progress, results/history, and supported resume behavior preserve the selected population and provenance.
- Existing scenarios' defaults and named-dataset behavior do not change.
- Focused scenario/service tests use local fixtures and fake targets/scorers, with no paid calls.
Describe alternatives you've considered, if relevant
Do not create a GUI-specific batch executor, generate Python scenario classes from user text, add an unscored mode, or redesign memory. Avoid adding a duplicate global prompt-sending factory merely to work around baseline semantics.
Additional context
Start with Scenario, DatasetAttackConfiguration, MatrixAtomicAttackBuilder, AttackTechniqueRegistry, ScorerRegistry, and existing matrix-shaped scenarios as patterns. The core technique catalog intentionally omits bare prompt sending because it is normally a baseline; handle the custom scenario's default deliberately. Read doc\code\framework.md and scenario/setup-technique/model/test instructions.
- 主要言語
- Python
- スター
- 4.5k
- フォーク
- 896
- 平均マージ
- 3日 5時間
- マージ済み PR（30日）
- 200
環境構築
このプロジェクトの環境構築ファイルはまだ確認していません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
microsoft/PyRIT のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 86/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 84/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 1 日以内に返信
microsoft/PyRIT の issue をすべて見る
似ている issue
-
good first issue
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
vllm-project/vllm-metal#822 ·
メンテナーはふだん 1 日以内に返信
-
vector-store
難易度 1/5 1〜3時間 初心者へのやさしさ 90/100
メンテナーはふだん 1 日以内に返信
-
[Bug]: chunk_span_bounds and _validated_chunk_spans reject Pydantic models ChunkSpan and AudioFileオープン
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
BasedHardware/omi#19047 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
メンテナーはふだん 1 日以内に返信