Four dataset tests fail on Windows because fixtures hard-code POSIX paths

Open Beginner friendly
  • Is something wrong?
#679 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing

Research direction

Start with the named tests in tests/datasets/test_cityscapes.py, tests/datasets/test_nuimages.py, and tests/datasets/test_semantickitti.py, and inspect their fake path and filesystem helpers. Run python -m pytest -q tests/datasets on Windows and Linux; update only the fixtures so their path comparisons work on both platforms. Done means all 93 dataset tests pass on both systems.

Written by the indexing model from the issue text.

Description

Summary

On Windows, 4 tests under tests/datasets/ fail while the remaining 89 pass. The library code behaves correctly; the failures come from the test fixtures, which hard-code POSIX-style fake paths (/fake/...) and compare them as plain strings against values the library has normalised with os.path.abspath / os.path.join (which produce C:\fake\... with backslashes on Windows). CI only runs on ubuntu-latest, so this is never exercised.

Environment
  • Windows 11, Python 3.11.9, PyTorch 2.4.1+cpu, no TensorFlow
  • PerceptionMetrics master @ 4a1ae41, installed with pip install -e .
  • python -m pytest -q → 89 passed, 4 failed
Failing tests and why
Test Assertion Cause
tests/datasets/test_cityscapes.py::test_build_dataset (L71) "frankfurt_000000_000294" in dataset → dataset is empty _fake_glob checks f"/{split}/" in pattern, but cityscapes.build_dataset builds the pattern with os.path.join after abspath (cityscapes.py L95, L112), so on Windows the pattern contains \val\ and never matches. _fake_exists compares against the POSIX _FAKE_LABEL string for the same reason.
tests/datasets/test_cityscapes.py::test_cityscapes_dataset (L127) dataset.dataset_dir == "/fake/cityscapes" actual value is C:\fake\cityscapes after abspath.
tests/datasets/test_nuimages.py::test_nuimages_segmentation_dataset (L79) label.endswith("generated/labels/v1.0-mini/sample_data_1.png") nuimages.py L129/L151 join with os.path.join, giving generated/labels\v1.0-mini\sample_data_1.png (mixed separators).
tests/datasets/test_semantickitti.py::test_build_dataset (L114 → semantickitti.py L167) AssertionError: Dataset directory not found build_dataset calls os.path.abspath(dataset_dir) first (L165), so _fake_isdir receives C:\fake\SemanticKITTI and path == _FAKE_ROOT is False.
Reproduce
git clone https://github.com/JdeRobot/PerceptionMetrics && cd PerceptionMetrics
python -m venv .venv && .venv\Scripts\activate
pip install -e . pytest
pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pytest -q tests/datasets
Proposed fix

Keep the library untouched and make the fixtures platform-agnostic: normalise the fake paths/patterns before comparing (e.g. os.path.normpath / abspath on the expected constants, and pattern.replace("\\", "/") inside the fake glob / isdir / exists helpers). No skipif on Windows — the tests should actually run there.

I have a small test-only patch (3 files under tests/datasets/, +42/−18) that does exactly this. With it the suite is 93 passed on Windows, and still 93 passed on Linux (WSL Ubuntu 24.04) — on POSIX os.path.abspath("/fake/...") returns the same string as before, so the Linux behaviour of the fixtures is unchanged by construction. Happy to open a PR if this approach is OK with you.

Optionally, adding a windows-latest entry to the CI matrix in .github/workflows/ci-run-tests.yml would catch regressions like this in the future.

Dominant language
Python
Stars
115
Forks
117
Avg merge
1d 14h
Merged PRs (30d)
12

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JdeRobot/PerceptionMetrics

All issues in JdeRobot/PerceptionMetrics

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.