Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Tests that import through a linked directory test the original file, so mutants are reported SURVIVED (Python: tests/../src)

Open
  • Is something wrong?
#19 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python, typescript
Domain
devtools, testing

Research direction

Start by tracing how worker directories are prepared and linked, then follow the test command execution path for Python and TypeScript. Reproduce the Python case with the provided conftest.py and pytest command, and compare worker behavior with copied test directories. Done means tests run against mutated files in the worker and mutation verdicts match the in-place recheck; the issue leaves the fix design open.

Written by the indexing model from the issue text.

Description

In a worker, every directory except the mutated file's own directory chain is a symlink back to the project. When a test reaches its code through a relative path that climbs out of a linked directory (tests/../src), the OS resolves .. from the link target, so the test imports the original, unmutated file. Every such mutant is reported SURVIVED while the tests do kill it.

Repro (mutator a5a0cb8, Python 3.12, pytest 9.1.1, macOS 26)

pyproject.toml:

[project]
name = "demo"
version = "0"

src/calc.py:

def big(x):
    return x > 10

tests/conftest.py:

import os, sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))

tests/test_calc.py:

from calc import big

def test_eleven_is_big():
    assert big(11)

def test_ten_is_not_big():
    assert not big(10)

$ python -m pytest -q tests
2 passed
$ mutator --no-coverage --test-command "/abs/path/python -m pytest -q tests" src/calc.py
SURVIVED  src/calc.py:2 > -> >=
big      calc            0         1          0      1   0.0%

Applying the same mutant by hand (x >= 10) and running the tests in place gives 1 failed, 1 passed, so it should be KILLED.

In the worker tests is a link to the project's tests, so tests/../src is the project's src, not the worker's.

Where I hit it

A project whose conftest.py adds tests/../scripts to sys.path scored 7.4% with mutator. With tests copied into each worker instead of linked, the same run scored 58.3%, and spot checks in place agreed with every verdict.

Same cause, opposite symptom (TypeScript)

Vitest resolves a linked test file to its real path outside the worker and does not load it, so the test command fails and every mutant is reported KILLED (100%). Copying the test folder into the worker fixed that run too.

Workaround

Wrap the test command so the folders the tests climb through are replaced by real copies in the worker before the tests run. A small in-place recheck (apply 2 killed and 2 survived mutants directly in the project, compare with mutator's verdict) catches either symptom.

A fix could be to copy rather than link the test directories, or to offer an option listing directories to copy into each worker.

Dominant language
Python
Stars
55
Forks
7
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from unclebob/mutator

All issues in unclebob/mutator

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.