ExactTextMatching reports a match for an empty or whitespace-only target
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
从 pyrit/analytics/text_matching.py 中的 ExactTextMatching.is_match() 开始，然后对比 tests/unit/analytics/test_text_matching.py 中现有的 test_target_too_short 覆盖范围。为为空和仅包含空白字符的 target 添加覆盖，同时保留非空 target 的匹配行为，并运行聚焦的 text-matching 测试。当空 target 不再匹配且现有行为保持不变时，即表示完成。
由索引模型根据 Issue 内容生成。
描述
Describe the bug
ExactTextMatching.is_match() guards an empty text (if not text: return False) but not an empty target, and "" in anything is True in Python. With the default ignore_whitespace=True the check is reachable with a whitespace-only target too, because target.strip() turns it into "".
Its sibling in the same module, ApproximateTextMatching, has the opposite behaviour and says why:
if len(target) < self._n:
return 0.0 # Confidence is too low for short targets
That rule is pinned by tests/unit/analytics/test_text_matching.py::test_target_too_short. So the two implementations of the same TextMatching interface disagree on a degenerate target.
This matters beyond the helper: DecodingScorer defaults to ExactTextMatching(case_sensitive=False) and calls it with no guard on user_piece.original_value and user_piece.converted_value — while guarding the adjacent decoded_text on the very next lines with if decoded_text and .... MessagePiece.converted_value defaults to "", so when no converter ran, the converted_value check matches every response and the scorer reports a successful decoding.
Steps/Code to Reproduce
from pyrit.analytics import ApproximateTextMatching, ExactTextMatching
print(ExactTextMatching().is_match(target="", text="I refuse to help with that."))
print(ExactTextMatching().is_match(target=" \n ", text="I refuse to help with that."))
print(ExactTextMatching(case_sensitive=True).is_match(target="", text="anything"))
print(ExactTextMatching(ignore_whitespace=False).is_match(target="", text="hello"))
print(ApproximateTextMatching().is_match(target="", text="hello world")) # sibling
Expected Results
The first four should be False: a target that carries no content cannot be found in the text. The sibling already returns False. is_match(target="refuse", text="I refuse to help") should stay True.
Actual Results
True
True
True
True
False
Screenshots
Not applicable.
Versions
- OS: macOS
- Python version: 3.11
- PyRIT version: installed from
mainin editable mode (pyrit/analytics/text_matching.py) - Existing coverage:
tests/unit/analytics/test_text_matching.py::test_empty_textcovers an empty text only; no test covers an empty target.
Proposed direction
I would add the symmetric guard to ExactTextMatching.is_match — return False when the (whitespace-normalised) target is empty — and a test mirroring test_target_too_short. That keeps the fix in the shared abstraction, so DecodingScorer and any other caller are covered without touching them. Happy to send a PR if that matches your intent; I am also happy to guard the two call sites in DecodingScorer instead if you prefer the narrower change.
- 主要语言
- Python
- 星标
- 4.5k
- 派生
- 896
- 平均合并
- 3 天 2 小时
- 30 天内合并 PR
- 210
环境准备
我们还没有检查这个项目的环境配置文件。先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
microsoft/PyRIT 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
microsoft/PyRIT#2888 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复
-
Bug: triage GUI help wanted
难度 2/5 1-3 小时 新手友好度 86/100
microsoft/PyRIT#2868 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
相似的 Issue
-
docs pydanty:is-working
难度 2/5 1-3 小时 新手友好度 75/100
pydantic/pydantic-ai#8863 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
run-llama/llama_index#23278 ·
维护者通常 2 天内回复
-
documentation from-review-extraction github-actions priority: low severity:nit
难度 1/5 1 小时以内 新手友好度 92/100
LearningCircuit/local-deep-research#6946 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
oracle/langchain-oracle#323 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 88/100
tenstorrent/tt-metal#58057 · 1 条评论 ·
维护者通常 1 天内回复