BUG TAP/PAIR skip their first turn after a prepended conversation (no adversarial system prompt, next_message ignored)
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 68/100
Línea de trabajo
Start by tracing TAP and PAIR through _TreeOfAttacksNode._is_first_turn(), then inspect the mock-target scenario described in the issue. Done means a prepended conversation does not suppress the first turn: the adversarial system prompt is set, next_message is used, and the scorer can receive the initial prompt instead of both attacks reporting FAILURE.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
If you pass a prepended_conversation to TAP or PAIR, the first turn never happens properly.
_TreeOfAttacksNode._is_first_turn() checks whether the objective conversation in memory is empty, but the prepended messages are written into that same conversation. So after any prepend it's never the first turn, which means:
set_adversarial_system_promptnever runs, so the adversarial chat has no system prompt at all, and the prepended context is never usednext_messageis silently ignored- with a system-only prepend, the subsequent-turn path raises "Cannot proceed without an assistant response", every node gets pruned, and the attack reports FAILURE without sending anything
With mock targets, a scorer that would succeed right away, and just a system message prepended:
TreeOfAttacksWithPruningAttack: FAILURE | prompts sent to target: [] | adversarial system prompts set: 0
PAIRAttack: FAILURE | prompts sent to target: [] | adversarial system prompts set: 0
Fix: don't count the prepended messages when deciding whether it's the first turn.
- Lenguaje dominante
- Python
- Estrellas
- 4.5k
- Forks
- 896
- Merge medio
- 3 d 8 h
- PR fusionados (30 d)
- 191
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/PyRIT
-
BUG HarmBench loader drops ContextString, so contextual behaviors are sent without their context Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
Todos los issues de microsoft/PyRIT
Issues similares
-
agent-ready documentation needs-triage
Dificultad 1/5 1-3 horas Aptitud para principiantes 88/100
-
documentation
Dificultad 1/5 Menos de una hora Aptitud para principiantes 91/100
-
workflow-status page template still says reusable workflows are "triggered only by workflow_call:" Abierto
Dificultad 1/5 Menos de una hora Aptitud para principiantes 92/100
-
Add https://search.jeremyh.xyz/ Abiertoinstance instance add
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
searxng/searx-instances#939 · 1 comentario ·
-
area-deployment area-integrations triage:bot-seen
Dificultad 2/5 Medio día Aptitud para principiantes 86/100