FEAT: Support custom scenarios over selected dataset examples
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
Línea de trabajo
Keep this work blocked until prerequisites 12–13 are promoted. Start with Scenario, DatasetAttackConfiguration, MatrixAtomicAttackBuilder, AttackTechniqueRegistry, ScorerRegistry, and existing matrix-shaped scenarios; read doc\code\framework.md and the scenario/setup-technique/model/test instructions. Done means focused local-fixture tests cover exact selected populations, validation, shared estimates and launch configuration, provenance, progress, results/history, and resume behavior without changing existing scenarios.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Is your feature request related to a problem? Please describe.
The explorer's multi-select flow needs an ordinary scenario over exactly the selected examples, defaulting to prompt sending. Reusing another scenario's defaults could unexpectedly add techniques, sample away selections, or run prompt sending twice as both a technique and baseline.
Work item 14 of 17 in #2744. Technical prerequisites: items 12-13. Keep not ready yet until promoted.
Describe the solution you'd like
Add the smallest registered scenario/configuration adapter needed for explicit selected examples. Reuse Scenario, existing attack techniques/factories, atomic attack construction, scorer registry resolution, and normal scenario progress/results/history.
- The default effective attack is exactly prompt sending with no additional converters. Do not fall back to another scenario's default technique set or emit a duplicate baseline for the same population.
- Accept a compatible configured objective-scorer preset, defaulting to the compatible configured default. Require valid objectives; preserve source objectives and accept explicit user-supplied objectives for examples that need them instead of inventing goals.
- Resolve explicit selections through the contract from item 13. Retain group boundaries, roles, multimodal content, and provenance. Explain unsupported seed shapes rather than flattening templates or simulated configuration into text.
- Expose supported registered techniques and converter-composition seams for the later UI. Attacks/converters/scorers keep their normal responsibilities; the scenario packages them and owns execution orchestration.
- Use registry-compatible construction and metadata introspection without fetching datasets or generating conversations. Validate execution prerequisites at the appropriate run/setup boundary.
Acceptance criteria:
- N valid selected examples with default settings produce exactly N prompt-sending executions, not N plus an extra baseline/default attack set.
- Missing objectives/scorers, incompatible modalities/techniques, and invalid groups produce actionable validation errors.
- Read-only estimates and confirmed launch use the same explicit selection and effective configuration.
- Normal scenario progress, results/history, and supported resume behavior preserve the selected population and provenance.
- Existing scenarios' defaults and named-dataset behavior do not change.
- Focused scenario/service tests use local fixtures and fake targets/scorers, with no paid calls.
Describe alternatives you've considered, if relevant
Do not create a GUI-specific batch executor, generate Python scenario classes from user text, add an unscored mode, or redesign memory. Avoid adding a duplicate global prompt-sending factory merely to work around baseline semantics.
Additional context
Start with Scenario, DatasetAttackConfiguration, MatrixAtomicAttackBuilder, AttackTechniqueRegistry, ScorerRegistry, and existing matrix-shaped scenarios as patterns. The core technique catalog intentionally omits bare prompt sending because it is normally a baseline; handle the custom scenario's default deliberately. Read doc\code\framework.md and scenario/setup-technique/model/test instructions.
- Lenguaje dominante
- Python
- Estrellas
- 4.5k
- Forks
- 896
- Merge medio
- 2 d 19 h
- PR fusionados (30 d)
- 206
Preparar el entorno
Aún no hemos revisado los archivos de configuración de este proyecto. Empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/PyRIT
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
microsoft/PyRIT#2888 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 91/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 1 día
-
Bug: triage GUI help wanted
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
microsoft/PyRIT#2868 · 1 comentario ·
Los mantenedores suelen responder en 1 día
Todos los issues de microsoft/PyRIT
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-2 días Aptitud para principiantes 70/100
-
FingerprintSplitter raises ZeroDivisionError when int(frac_train * len(dataset)) floors to zeroAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Los mantenedores suelen responder en 7 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
lmstudio-ai/mlx-engine#376 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
pyiron/bagofholding#166 ·