ExactTextMatching reports a match for an empty or whitespace-only target
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 91/100
Research direction
Start in pyrit/analytics/text_matching.py at ExactTextMatching.is_match(), then compare the existing test_target_too_short coverage in tests/unit/analytics/test_text_matching.py. Add coverage for empty and whitespace-only targets while preserving matching for non-empty targets, and run the focused text-matching tests. Done means empty targets no longer match and the existing behavior remains intact.
Written by the indexing model from the issue text.
Description
Describe the bug
ExactTextMatching.is_match() guards an empty text (if not text: return False) but not an empty target, and "" in anything is True in Python. With the default ignore_whitespace=True the check is reachable with a whitespace-only target too, because target.strip() turns it into "".
Its sibling in the same module, ApproximateTextMatching, has the opposite behaviour and says why:
if len(target) < self._n:
return 0.0 # Confidence is too low for short targets
That rule is pinned by tests/unit/analytics/test_text_matching.py::test_target_too_short. So the two implementations of the same TextMatching interface disagree on a degenerate target.
This matters beyond the helper: DecodingScorer defaults to ExactTextMatching(case_sensitive=False) and calls it with no guard on user_piece.original_value and user_piece.converted_value — while guarding the adjacent decoded_text on the very next lines with if decoded_text and .... MessagePiece.converted_value defaults to "", so when no converter ran, the converted_value check matches every response and the scorer reports a successful decoding.
Steps/Code to Reproduce
from pyrit.analytics import ApproximateTextMatching, ExactTextMatching
print(ExactTextMatching().is_match(target="", text="I refuse to help with that."))
print(ExactTextMatching().is_match(target=" \n ", text="I refuse to help with that."))
print(ExactTextMatching(case_sensitive=True).is_match(target="", text="anything"))
print(ExactTextMatching(ignore_whitespace=False).is_match(target="", text="hello"))
print(ApproximateTextMatching().is_match(target="", text="hello world")) # sibling
Expected Results
The first four should be False: a target that carries no content cannot be found in the text. The sibling already returns False. is_match(target="refuse", text="I refuse to help") should stay True.
Actual Results
True
True
True
True
False
Screenshots
Not applicable.
Versions
- OS: macOS
- Python version: 3.11
- PyRIT version: installed from
mainin editable mode (pyrit/analytics/text_matching.py) - Existing coverage:
tests/unit/analytics/test_text_matching.py::test_empty_textcovers an empty text only; no test covers an empty target.
Proposed direction
I would add the symmetric guard to ExactTextMatching.is_match — return False when the (whitespace-normalised) target is empty — and a test mirroring test_target_too_short. That keeps the fix in the shared abstraction, so DecodingScorer and any other caller are covered without touching them. Happy to send a PR if that matches your intent; I am also happy to guard the two call sites in DecodingScorer instead if you prefer the narrower change.
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 896
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 214
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/PyRIT
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 2 days
-
Bug: triage GUI help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
microsoft/PyRIT#2868 · 1 comment ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 2 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
PedestrianDynamics/pyFDS-Evac#343 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
theskumar/python-dotenv#708 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
Docs Timedelta
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pandas-dev/pandas#69919 ·
Maintainers usually reply within 1 day
-
API documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
zephyrproject-rtos/west#1009 · 2 comments ·
Maintainers usually reply within 3 days