BUG PromptInject run configuration shows goal text as the technique
まだ誰も着手していません。
評価
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 初心者へのやさしさ
- 65/100
- issue の種類
- バグ
- 明瞭さ
- 明確に書かれている
- 活発さ
- 活発
- 技術スタック
- fastapi, javascript, postgresql, python, react, sqlite, typescript
調査の方向性
The bug is in the backend's scenario_run_service.py where techniques_used is populated from group.display_group instead of the canonical technique name. Start by examining lines 1382-1386, 1583-1586, and 1985-2007 in that file. The frontend displays this field in ScenarioRunPage.tsx lines 413-415. Reproduce the issue locally by running garak.prompt_inject with a custom goal_text and verify the mismatch. The fix should ensure the technique identity is used consistently across the run summary, list projection, and progress response.
索引モデルが issue の本文から書いたものです。
説明
Describe the bug
After running garak.prompt_inject, Run configuration > Techniques shows the requested goal text instead of the selected technique. The preview and the separate Techniques section show the correct technique, so the saved-run view contradicts them.
For example, selecting ignore_print with goal_texts=["AUDIT_SAFE_MARKER"] produces a completed result whose Run configuration lists AUDIT_SAFE_MARKER as the technique.
This makes the result summary misleading when reviewing or sharing which method was tested. Execution and Resume preserved the correct technique and custom goal in the audit; the observed defect is in the API/UI summary.
Steps/Code to Reproduce
- Start the GUI/backend and configure a local OpenAI-compatible test target. A deterministic local provider is sufficient; no paid provider or external dataset is needed.
- Open Scanner > garak.prompt_inject and select that target.
- Replace the default
goal_textswithAUDIT_SAFE_MARKER. - Leave only
ignore_printselected, set Max dataset size to1, and leave baseline disabled. - Click Launch scan. The preview correctly lists
ignore_printunder Techniques and the marker undergoal_texts. - Confirm the launch and open the completed run.
- Compare Run configuration > Techniques with the separate Techniques section.
- Reload the page. The mismatch remains.
The relevant POST /api/scenarios/runs payload is:
{
"scenario_name": "garak.prompt_inject",
"target_name": "<local test target registry name>",
"techniques": ["ignore_print"],
"include_baseline": false,
"max_dataset_size": 1,
"max_retries": 0,
"scenario_params": {
"goal_texts": ["AUDIT_SAFE_MARKER"]
}
}
This also reproduced with a second harmless marker, AUDIT_RESUME_SAFE, after a deliberately failed local run was successfully resumed under the same run ID.
Expected Results
Run configuration > Techniques and the API's techniques_used should identify ignore_print. Goal-based labels can remain under Atomic attack groups; they are not technique identities.
Actual Results
| Location | Value |
|---|---|
Submitted techniques |
["ignore_print"] |
Submitted scenario_params.goal_texts |
["AUDIT_SAFE_MARKER"] |
| Run configuration > Techniques | AUDIT_SAFE_MARKER |
Progress API run.techniques_used |
["AUDIT_SAFE_MARKER"] |
| Plan's technique identity and separate Techniques section | ignore_print |
The wrong summary survives reload and also appears after Resume. The separate Techniques section is a workaround for identifying the actual method.
The source points to the backend projection:
- PromptInject groups atomic attacks by goal text, which is valid grouping behavior.
- The service fills
techniques_usedfromgroup.display_groupin the run summary, list projection, and progress response. - The frontend displays that field as Techniques.
The projection should use the canonical technique identity consistently across these responses, rather than renaming the legitimate goal-based groups. Regression coverage should include a run where display_group differs from technique_name.
Screenshots
Not attached. The request/response comparison above records the observed mismatch from two independent runs in the September 23 frontend audit.
Versions
- Tested commit:
5453025c128d36a4ac624c10097a5e85031302ecfrommain. - PyRIT: editable
1.2.0.dev0;garak.prompt_injectversion 3. - Browser: Chromium through Playwright 1.63.0, desktop viewport 1440 x 900.
- Backend: isolated in-memory database and loopback-only synthetic provider.
pyrit.show_versions() output, with the local executable path redacted:
System:
python: 3.14.4 (main, Apr 14 2026, 14:30:57) [MSC v.1944 64 bit (AMD64)]
executable: <worktree>\.venv\Scripts\python.exe
machine: Windows-11-10.0.26200-SP0
Python dependencies:
pyrit: 1.2.0.dev0
Cython: None
numpy: 2.4.6
openai: 2.54.0
packaging: 25.0
pip: None
scipy: 1.17.1
setuptools: 83.0.0
sqlite3: None
torch: 2.14.0
transformers: 5.17.0
- 主要言語
- Python
- スター
- 4.5k
- フォーク
- 896
- 平均マージ
- 3日 8時間
- マージ済み PR(30日)
- 191
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
microsoft/PyRIT のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
-
難易度 3/5 1〜2日 初心者へのやさしさ 68/100
microsoft/PyRIT の issue をすべて見る
似ている issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
stephrobert/dsoxlab#238 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
sublimehq/package_control#1780 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
nwg-piotr/nwg-displays#145 ·