mllam/neural-lam

Author notebooks as jupytext-paired `py:percent` sources instead of committing raw `.ipynb`

Open

#708 创建于 2026年7月16日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Python (276 fork)auto 404
cicddocumentationgood first issuehelp wanted

仓库指标

Star
 (282 star)
PR 合并指标
 (PR 指标待抓取)

描述

docs/notebooks/create_reduced_meps_dataset.ipynb (and the DANRA tutorial coming in #577) are committed as raw .ipynb. That gives us:

  • unreviewable diffs: a one-line code change shows up as a JSON blob, so notebook changes effectively go unreviewed;
  • executed outputs committed into git, including absolute local paths from whoever ran it last;
  • notebook code that no linter ever sees, and nothing that detects when it drifts from the library.

There is no jupytext/myst tooling in the repo today (nothing in pyproject.toml or .pre-commit-config.yaml).

I suggest to make a py:percent file the source of truth and keep it paired with the .ipynb via a pre-commit hook, i.e. the same self-consistency contract as the graph-storage-spec hook from #323 @leifdenby @observingClouds:

- id: jupytext-sync
  name: Keep paired notebooks in sync with their py:percent source
  entry: jupytext --sync
  language: python
  additional_dependencies: [jupytext]
  files: ^docs/notebooks/.*\.(ipynb|py)$

The .py is diffable, reviewable and carries no outputs. Two practical notes from trying it. But if you can fix them instead of ignoring them, even better.

  • the generated .py is prose-heavy (markdown cells become long comments), so it needs excluding from black/isort/flake8/mypy;
  • it also needs excluding from trailing-whitespace/end-of-file-fixer, which otherwise strip whitespace jupytext regenerates and the two hooks fight forever.

This pairs naturally with giving notebooks a Python API to call (see https://github.com/mllam/neural-lam/issues/707): the source then reads as plain, reviewable Python instead of subprocess calls plus path globbing.

贡献者指南