mllam/neural-lam

Add property-based testing with hypothesis

Open

#602 opened on Apr 15, 2026

View on GitHub
 (9 comments) (0 reactions) (1 assignee)Python (276 forks)auto 404
cicdgood first issuehelp wanted

Repository metrics

Stars
 (282 stars)
PR merge metrics
 (PR metrics pending)

Description

Most tests currently use handpicked input values and assert specific outputs. Hypothesis generates inputs automatically and finds edge cases you wouldn't think to write. E.g. https://github.com/ecmwf/anemoi already uses it for testing attention and MLP layers.

Candidates

  • Standardization: output should always have mean ~0 and std ~1 regardless of input shape or value range
  • Clamping: model output should always stay within declared bounds for any input tensor. test_clamping.py already tests specific cases, hypothesis would test the invariant more thoroughly
  • Dataset slicing: test_time_slicing.py hardcodes specific input/output pairs - these could be expressed as properties that hold for any valid combination of ar_steps, num_past_forcing_steps, num_future_forcing_steps
  • Datastore interface contracts: get_dataarray output shape should always match num_grid_points for any valid split and category

Setup

hypothesis would be a dev dependency only and integrates directly with pytest via @given decorators, so no CI changes are needed.

Contributor guide

Research direction
Identify invariant properties for each candidate (standardization, clamping, dataset slicing, datastore contracts) and write Hypothesis tests using the @given decorator. Focus on edge cases and input variations.
Tech stack
python
Domain
testing qa
Issue type
Test
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Pythonpytest
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75