chore(test): trial a mutation testing tool (gremlins or go-mutesting) after the property tests land

オープン
#222 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
1週間以上
初心者へのやさしさ
15/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
github-actions, go
領域
testing

調査の方向性

This is blocked by #221, so wait for its property and fuzz tests to merge before starting. Once they have, run each candidate tool over internal/indexer and internal/graphstore, then over the changed lines of five real PRs, and classify every LIVED mutant. Done means the counts and the adoption decision are recorded in this issue, with mise.toml and mise.lock updated only if a tool is adopted.

索引モデルが issue の本文から書いたものです。

説明

area:ci chore priority:low
What kind of chore?

Test infrastructure

What needs doing?

After the property and fuzz tests from #221 land, trial a mutation testing tool. Adopt it only when the trial meets the entry test below. #221 deletes the repo's own scripts/mutation_check.py, its unit tests, task mutation-check and the mutation-check skill. Nothing replaces them until this trial decides.

Candidates

Both are active. Facts as of 2026-10-31:

go-gremlins/gremlins avito-tech/go-mutesting
latest release v0.6.0, 2025-12-06 v2.3.1, 2025-12-26
install release tarballs with a checksums file, so mise.lock records them no release assets, go install only
Go required 1.25 1.27. The repo pins 1.26.10 with GOTOOLCHAIN=local, so this needs a toolchain decision first
scope to what changed diff: mutates only changed lines, and coverage guidance skips uncovered lines targets are files or packages, no line-level diff
how tests run on a copy, the tree is untouched replaces the original file in place, then restores it
CI gate threshold.efficacy and threshold.mutant-coverage a printed mutation score, parsed by the caller
mutators 11, 5 on by default. No statement or branch removal a larger set: branch, statement removal, loop break, numbers, expression
suppression exclude-files inline mutator-disable-next-line comments in product source
license Apache-2.0 MIT

Neither is assumed. The trial runs both over the same packages and the same PRs.

Entry test
  1. Wait for the sync-equals-reindex property and the keyenc fuzz target from #221.
  2. Run each candidate once over internal/indexer and internal/graphstore. Record killed, lived, not covered, not viable, and minutes per run.
  3. Run each candidate over the changed lines of the next five real PRs. For go-mutesting, pass the changed files as targets. Record the same counts and minutes.
  4. Read every LIVED line. Mark each one as one of three kinds. A real gap means a reviewer agrees a test is missing. Noise means an equivalent mutant. A generator gap means a property does not reach the line.

To adopt a tool as a CI gate, both of these must hold:

  • At least one surviving mutant was a real gap or a generator gap that led to a stronger test.
  • The per-PR cost on changed lines stayed under a few minutes.

If neither tool meets that, the outcome is still useful. Record the counts in this issue, close it, and leave the tool as something anyone can run locally.

If adopted
  • Pin the tool in mise.toml. For gremlins, as a GitHub release with mise.lock checksums. For go-mutesting, resolve the Go 1.27 requirement first.
  • Run it on changed lines only, through paths-filter from #221, never over the whole module on every push.
  • The gate asserts that it mutated at least one line (rule 84d1gfpywd). A run that mutates nothing fails.
  • Rewrite the fix PR template's "I watched it fail before the fix" proof to accept the tool's report instead of pasted output.
  • Put no tool-specific disable comments in product source.
Why now?

Not now. This issue exists so the deletion in #221 has a named successor and an entry test, instead of a silent gap. When part 3 of #221 (property and fuzz tests) merges, this trial starts.

Does this touch anything load-bearing?
  • The release pipeline (release.yml, .goreleaser.yaml, signing)
  • The upgrade identity (internal/upgrade/verify.go)
  • The perf baseline (tools/bench/baseline.json)
  • The graph schema / meta-key layout
  • None of these
Before submitting
  • I searched existing issues and this is not a duplicate

Blocked by #221. Related: #176 and #182 (the skill this replaces).

🤖 Generated with Claude Code

https://claude.ai/code/session_0167DcypPuyLUHrwdrVKYYG8

主要言語
Go
スター
0
フォーク
0
平均マージ
7時間 42分
マージ済み PR(30日)
111

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

seanb4t/codegraph-go のほかの issue

seanb4t/codegraph-go の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。