FEAT expose mask parameter on OpenAIImageTarget for inpainting support
@romanlutz ya está trabajando en esto.
Desde el 15/6/2026.
Evaluación
Este issue todavía no se ha evaluado.
Descripción
Summary
OpenAIImageTarget._send_edit_request_async currently calls client.images.edit(**image_edit_args) with only image, prompt, size, and a few formatting flags. The OpenAI SDK's images.edit endpoint also accepts an optional mask parameter — a PNG whose fully-transparent regions indicate the only pixels the model is permitted to modify, while every other pixel must be returned unchanged. This is inpainting, and it's the standard primitive for "keep this scene but regenerate just the masked subregion."
PyRIT does not expose mask today, so red-teamers using OpenAIImageTarget for image-editing attacks can only do whole-image edits. They cannot pin which subregion of the input image is allowed to change, which loses an important degree of control for evaluations that depend on precisely localized edits.
Proposed change
- Add a new
MessagePiecedata type or convention for image masks (options: a dedicatedimage_maskdata type, or reuseimage_pathwith a metadata flag — e.g. prompt_metadata={"is_mask": True}). - In OpenAIImageTarget._send_edit_request_async, separate the input pieces into base image(s) and mask. When a mask is present, add
"mask": (name, bytes, mime_type)toimage_edit_args. - Validate that mask is only attached when exactly one base image is provided (the OpenAI endpoint only supports masking a single base image — multi-image edits with a mask would need to be rejected with a clear error).
- Update the multimodal/router components (e.g. ModalityFeedbackRouter.build_objective_input_message) so attacks that want to forward a mask alongside the adversarial-generated text + base image can do so without rebuilding the message from scratch.
- Tests:
- Mask piece is correctly serialized and passed through to the SDK call.
- Multi-image + mask raises a validation error.
- Backward compatibility: existing single-image and multi-image edit flows (no mask) continue to work unchanged.
API reference
OpenAI image edit endpoint: https://platform.openai.com/docs/api-reference/images/createEdit — see the mask parameter.
Scope / non-goals
- No automatic mask generation from text (e.g. "mask the face") — masks are provided by the caller.
Acceptance criteria
- A caller can construct a
Messagewith[text, image_path, mask]and OpenAIImageTarget.send_prompt_async routes the mask correctly toclient.images.edit(..., mask=...). - The unmasked portions of the input image are pixel-for-pixel preserved (verified in an integration test against a fixture image).
- Existing tests continue to pass.
- Lenguaje dominante
- Python
- Estrellas
- 4.5k
- Forks
- 896
- Merge medio
- 3 d 7 h
- PR fusionados (30 d)
- 155
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/PyRIT
-
Bug: triage help wanted
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
-
Bug: triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
-
BUG: ScorerMetrics.to_json() raises TypeError on the trial_scores array ScorerEvaluator attaches Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 78/100
-
Bug: triage help wanted
Dificultad 3/5 1-2 días Aptitud para principiantes 75/100
-
BUG Console printers write ANSI/OSC sequences from target responses straight to the terminal Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 68/100
Todos los issues de microsoft/PyRIT
Issues similares
-
documentation help wanted
Dificultad 2/5 1-3 horas Aptitud para principiantes 90/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 90/100
simonw/sqlite-utils#872 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100