Repository metrics
- Stars
- (282 stars)
- PR merge metrics
- (PR metrics pending)
Description
Running pytest with no arguments currently downloads a tarball from ECMWF's S3, unpacks it, computes standardization stats, builds graphs, and runs a full Lightning training loop. That's a lot to ask for a quick local check.
Beyond the slow feedback loop, mixing test types in a flat directory makes it harder to understand what each test is actually verifying. A reader can't tell at a glance whether a test is checking isolated logic or exercising the full stack, which makes the suite harder to maintain and extend.
Most tests call init_datastore_example() from conftest.py, which triggers the S3 download. Only a handful are actually fast and isolated:
- Unit tests (no I/O):
test_imports.py,test_config.py,test_cli.py,test_time_slicing.py, and thetest_all_gather_cat_*functions intest_training.py - Integration tests (need real data):
test_datastores.py,test_graph_creation.py,test_training.py::test_training,test_datasets.py,test_plotting.py,test_plot_graph.py
Options
- Register a marker in
pyproject.toml:
[tool.pytest.ini_options]
markers = [
"integration: tests that require network access or real data (deselect with '-m not integration')",
]
- Split into
tests/unit/andtests/integration/, movingconftest.py(and the S3 fixture) into the integration folder. Cleaner long-term, makes the distinction visible in the file tree, and prevents the fixture from being accidentally used in unit tests.
Bonus
test_clamping.py could become a true unit test by swapping the real MDPDatastore for the DummyDatastore that already exists in tests/dummy_datastore.py. Same logic, no S3 dependency. test_time_slicing.py already does this well and is a good model to follow.