Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

BUG: PlagiarismScorer accepts invalid n-gram size and blank reference text

未关闭 适合新手
  • 哪里不对？
#2,971 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 2 天内回复

@RohithPariki 已经在做这个了。

开始于 2026年10月3日。

  • #2972 来自 @RohithPariki —— 未关闭

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
85/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python
领域
security

调研方向

从 pyrit/score/float_scale/plagiarism_scorer.py 中的 PlagiarismScorer.init 开始，然后将其参数处理与 issue 中描述的 ApproximateTextMatching 验证进行比较。运行现有的 PlagiarismScorer 测试，并为无效的 n、空白参考文本和无效指标添加覆盖；完成标准是构造过程拒绝所列出的每种情况，同时不改变有效评分。

由索引模型根据 Issue 内容生成。

描述

Describe the bug

PlagiarismScorer does not validate its n (n-gram size) or reference_text parameters during initialization.

  1. n <= 0 creates false-positive 100% plagiarism matches:
    When n = 0, _ngram_set returns {()} (an empty tuple). In PlagiarismMetric.JACCARD, len(ref_ngrams & res_ngrams) / len(ref_ngrams) evaluates to 1 / 1 = 1.0 for any response. A completely unrelated prompt response is therefore reported as 100% plagiarized.
    When n < 0 (e.g. n = -1), negative slicing produces spurious n-grams and invalid partial overlap scores (e.g. 0.5).
    When n is a non-integer float (e.g. 2.5), construction succeeds, but scoring later crashes with TypeError: 'float' object cannot be interpreted as an integer.
    When n is a boolean (e.g. False), it evaluates as 0 and yields the same false-positive 1.0.

  2. Empty or whitespace-only reference_text silently reports 0.0 for all responses:
    When reference_text is "" or whitespace-only " ", _tokenize(reference) produces [] (reference_len = 0). The condition response_len == 0 or reference_len == 0 causes _plagiarism_score to silently return 0.0 for every response, even when the response is identical to the reference text. A static misconfiguration is thus silently swallowed and reported as non-plagiarism.

The sibling class in analytics, ApproximateTextMatching, already validates that its n-gram parameter n must be an integer >= 1.

Steps/Code to Reproduce

from pyrit.score import PlagiarismScorer, PlagiarismMetric

# 1. n=0 scores 1.0 (100% match) for completely unrelated text
scorer = PlagiarismScorer(
    reference_text="The quick brown fox jumps over the lazy dog",
    metric=PlagiarismMetric.JACCARD,
    n=0,
)
score = scorer._plagiarism_score(
    response="Completely unrelated content about astrophysics and quantum mechanics",
    reference="The quick brown fox jumps over the lazy dog",
    metric=PlagiarismMetric.JACCARD,
    n=0,
)
print("Score with n=0:", score)

# 2. n=-1 yields spurious partial match
print(
    "Score with n=-1:",
    scorer._plagiarism_score(
        response="Completely unrelated content",
        reference="The quick brown fox jumps over the lazy dog",
        metric=PlagiarismMetric.JACCARD,
        n=-1,
    ),
)

# 3. Empty reference_text silently evaluates every response as 0.0
empty_scorer = PlagiarismScorer(reference_text="", metric=PlagiarismMetric.LCS)
print("Score with empty reference:", empty_scorer._plagiarism_score(response="", reference=""))

Expected Results

PlagiarismScorer.__init__ should validate parameters at construction time and raise ValueError:

  • reference_text must be a non-empty string containing at least one word token.
  • n must be an integer >= 1 (rejecting 0, negative values, booleans, and non-integers).
  • metric must be an instance of PlagiarismMetric.

Actual Results

Score with n=0: 1.0
Score with n=-1: 0.5
Score with empty reference: 0.0

PlagiarismScorer.__init__ assigns self.reference_text = reference_text and self.n = n directly at pyrit/score/float_scale/plagiarism_scorer.py:54-56 without validation.

Versions

  • OS: Windows 11
  • Python: 3.14
  • PyRIT: main at c2ae5eeb
主要语言
Python
星标
4.6k
派生
924
平均合并
2 天 14 小时
30 天内合并 PR
227

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

  • 没有 Dockerfile 或 Docker Compose 文件
  • 有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

microsoft/PyRIT 的其他 Issue

查看 microsoft/PyRIT 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。