BUG: MemoryInterface.get_request_from_response fails with IndexError for non-zero starting sequence or sequence gaps
维护者通常 2 天内回复
评估
调研方向
The bug is in _execute_get_request_from_response in pyrit/memory/memory_interface.py, which indexes the conversation list by response.sequence - 1. Read _execute_get_conversation_messages to see how messages are ordered, then reproduce with the SQLite script in the issue. Pull request #3190 is already open against this issue, so check it before starting a new fix. Done means the method returns the message with the closest lower sequence, or raises a clear error when none exists.
由索引模型根据 Issue 内容生成。
描述
Describe the bug
In pyrit/memory/memory_interface.py, MemoryInterface._execute_get_request_from_response retrieves the request that produced a given assistant response:
def _execute_get_request_from_response(self, *, response: Message) -> Message:
if response.api_role != "assistant":
raise ValueError("The provided request is not a response (role must be 'assistant').")
if response.sequence < 1:
raise ValueError("The provided request does not have a preceding request (sequence < 1).")
conversation = self._dispatch_memory_operation(
"get_conversation_messages",
self._execute_get_conversation_messages,
conversation_id=response.conversation_id,
)
return conversation[response.sequence - 1]
The method treats response.sequence - 1 as a 0-based list index into conversation. However, get_conversation_messages returns messages ordered by sequence (sorted(conversation_by_sequence.keys())), where list index corresponds to sequence number only if the conversation started at sequence 0 and contains no sequence gaps.
Whenever:
- A conversation starts at a non-zero sequence (e.g. using
starting_sequenceinSeedPrompt.from_messagesorSimulatedConversationAttack), or - A conversation contains sequence gaps (e.g. after pruning or retries where pieces were deleted),
calling get_request_from_response (or get_request_from_response_async, or DecodingScorer which invokes this method) raises:
IndexError: list index out of range (or returns the wrong message if the index exists but does not align).
Steps/Code to Reproduce
import asyncio
from uuid import uuid4
from pyrit.memory import SQLiteMemory
from pyrit.models import MessagePiece
async def test_reproduce():
memory = SQLiteMemory(db_path=":memory:")
conversation_id = str(uuid4())
# Create a conversation with starting_sequence=10
pieces = [
MessagePiece(
role="user",
original_value="What is the capital of France?",
conversation_id=conversation_id,
sequence=10,
),
MessagePiece(
role="assistant",
original_value="Paris.",
conversation_id=conversation_id,
sequence=11,
),
]
await memory.add_message_pieces_to_memory_async(message_pieces=pieces)
conversation = await memory.get_conversation_messages_async(conversation_id=conversation_id)
response = conversation[1] # sequence=11
# Attempt to retrieve the request that produced the response
await memory.get_request_from_response_async(response=response)
asyncio.run(test_reproduce())
Expected Results
get_request_from_response_async returns the message immediately preceding the response in the conversation timeline (Message(role='user', sequence=10, ...)).
Actual Results
Traceback (most recent call last):
...
File "pyrit/memory/memory_interface.py", line 3041, in _execute_get_request_from_response
return conversation[response.sequence - 1]
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range
Suggested Fix
Instead of assuming conversation index equals sequence - 1, locate the preceding message in conversation with sequence < response.sequence:
conversation = self._dispatch_memory_operation(
"get_conversation_messages",
self._execute_get_conversation_messages,
conversation_id=response.conversation_id,
)
preceding_messages = [msg for msg in conversation if msg.sequence < response.sequence]
if not preceding_messages:
raise ValueError(
f"The provided request does not have a preceding request in conversation '{response.conversation_id}'."
)
return preceding_messages[-1]
- 主要语言
- Python
- 星标
- 4.6k
- 派生
- 944
- 平均合并
- 2 天 18 小时
- 30 天内合并 PR
- 358
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
microsoft/PyRIT 的其他 Issue
-
BUG: label queries only return the final conversation of an attack可能已有人在做 @u7k4rs6 于 2 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 85/100
microsoft/PyRIT#3095 · 4 条评论 · 已指派 1 人 ·
维护者通常 2 天内回复
-
BUG Configuration keeps runtime-status errors after polling recovers可能已有人在做 @joeyvictorino 于 6 天前认领。 未关闭Bug: triage GUI help wanted
难度 2/5 1-3 小时 新手友好度 86/100
microsoft/PyRIT#2868 · 6 条评论 ·
维护者通常 2 天内回复
-
Crescendo backtracking keeps the refused prompt when the objective target made a tool call可能已有人在做 @RachelWanggg 于 1 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 30/100
维护者通常 2 天内回复
-
BUG BinAsciiConverter UUencode output cannot be decoded when all words are converted可能已有人在做 @barkha1research-ai 于 1 天前认领。 未关闭Bug: triage
难度 2/5 1-3 小时 新手友好度 25/100
维护者通常 2 天内回复
-
FEAT Proposal: red-team decision models as targets (decision flip-rate testing)可能已有人在做 @shashank03-dev 于 1 天前认领。 未关闭
难度 5/5 一周以上 新手友好度 15/100
microsoft/PyRIT#3246 · 2 条评论 ·
维护者通常 2 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 65/100
-
难度 2/5 1-3 小时 新手友好度 72/100
UKGovernmentBEIS/control-arena#917 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 88/100
-
难度 2/5 1-3 小时 新手友好度 82/100
debpalash/VoiceStudio#2937 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
aws-samples/sample-per-model-bedrock#261 ·
维护者通常 1 天内回复