Disable explicit mypy tests?

Open
#10,934 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Stale
Tech stack
github-actions, python
Domain
ci-cd, testing

Research direction

Start by locating the explicit mypy tests introduced in PR #10038 and the GitHub CI jobs that run them. Review their current failures, runtime, and typing coverage; done means the fragile explicit tests are disabled without obscuring the remaining typing checks.

Written by the indexing model from the issue text.

Description

CI
What is your issue?

The explicit mypy tests introduced in https://github.com/pydata/xarray/pull/10038 seem to be quite fragile, e.g., they are currently broken at head and I know have been broken at head previously at least once. In neither case was there a real regression.

I see at least three issues with them:

  1. They seem to be particularly prone to version skew issues where they fail on GitHub but pass locally due to slight differences in environments. I think the current failure is due to mis-matched NumPy versions. (This might be improved when we switch to pixi.)
  2. They are really slow. Testing ~40 specific errors takes ~9 minutes on a single GitHub CI worker, which is not a great use of CI resources and clearly not scalable. Currently we use two CI workers just to test typing for the pipe method!
  3. They are inherently very fragile, checking for exact messages printed by mypy. Pretty much any package update causes error messages to change, which means these tests break, whether or not there is a real underlying issue.
Dominant language
Python
Stars
4.2k
Forks
1.5k
Avg merge
2d 21h
Merged PRs (30d)
74

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pydata/xarray

All issues in pydata/xarray

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.