mllam/neural-lam

Separate unit tests from integration tests

Open

#599 opened on Apr 15, 2026

View on GitHub
 (6 comments) (0 reactions) (1 assignee)Python (276 forks)auto 404
cicdgood first issuehelp wanted

Repository metrics

Stars
 (282 stars)
PR merge metrics
 (PR metrics pending)

Description

Running pytest with no arguments currently downloads a tarball from ECMWF's S3, unpacks it, computes standardization stats, builds graphs, and runs a full Lightning training loop. That's a lot to ask for a quick local check.

Beyond the slow feedback loop, mixing test types in a flat directory makes it harder to understand what each test is actually verifying. A reader can't tell at a glance whether a test is checking isolated logic or exercising the full stack, which makes the suite harder to maintain and extend.

Most tests call init_datastore_example() from conftest.py, which triggers the S3 download. Only a handful are actually fast and isolated:

  • Unit tests (no I/O): test_imports.py, test_config.py, test_cli.py, test_time_slicing.py, and the test_all_gather_cat_* functions in test_training.py
  • Integration tests (need real data): test_datastores.py, test_graph_creation.py, test_training.py::test_training, test_datasets.py, test_plotting.py, test_plot_graph.py

Options

  1. Register a marker in pyproject.toml:
[tool.pytest.ini_options]
markers = [
    "integration: tests that require network access or real data (deselect with '-m not integration')",
]
  1. Split into tests/unit/ and tests/integration/, moving conftest.py (and the S3 fixture) into the integration folder. Cleaner long-term, makes the distinction visible in the file tree, and prevents the fixture from being accidentally used in unit tests.

Bonus

test_clamping.py could become a true unit test by swapping the real MDPDatastore for the DummyDatastore that already exists in tests/dummy_datastore.py. Same logic, no S3 dependency. test_time_slicing.py already does this well and is a good model to follow.

Contributor guide