Testing is far too fragile
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 22/100
調査の方向性
The failures come from the image comparison in tests/integration/test_advanced_plot_examples.py and tests/integration/test_contour_plot_examples.py, which compare rendered plots against reference images in tests/reference-example-images. Run one failing case, such as test_example_1_basic_cylindrical, and read how the rms value is compared with the reference. The issue does not say whether the fix should be a tolerance change, regenerated references, or environment pinning, so a maintainer decision is needed first. Done means the suite passes across environments under the chosen approach.
索引モデルが issue の本文から書いたものです。
説明
Subject of the issue
I've just pulled the latest release, and 24 tests fail. They are all minor fails to do with image mismatch, but it's just too fragile.
================================================================= short test summary info ==================================================================
FAILED tests/integration/test_advanced_plot_examples.py::test_example_14_vector_with_colour_contour - Failed: Image mismatch for example 14: {'rms': np.float64(0.07727821087173495), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_15_polar_vector_plot - Failed: Image mismatch for example 15: {'rms': np.float64(1.2350842133667295), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_16b_basic_stream_plot - Failed: Image mismatch for example 16b: {'rms': np.float64(19.617677264399287), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_16c_enhanced_stream_plot - Failed: Image mismatch for example 16c: {'rms': np.float64(21.34025810212168), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_17_basic_stipple_plot - Failed: Image mismatch for example 17: {'rms': np.float64(0.21548615766477328), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_18_polar_stipple_plot - Failed: Image mismatch for example 18: {'rms': np.float64(0.14397116764560633), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_canari_1 - Failed: Image mismatch for example canari_1: {'rms': np.float64(0.02305860896416126), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/re...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_40_tracks_polar_stereographic - Failed: Image mismatch for example 40: {'rms': np.float64(0.01662284425174423), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_1_basic_cylindrical - Failed: Image mismatch for example 1: {'rms': np.float64(0.18352397386241887), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_3_map_limits_and_levels - Failed: Image mismatch for example 3: {'rms': np.float64(0.08961256746282698), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_4_north_pole_stereographic - Failed: Image mismatch for example 4: {'rms': np.float64(0.5533641260245302), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-...
FAILED tests/integration/test_contour_plot_examples.py::test_example_5_south_pole_with_boundary - Failed: Image mismatch for example 5: {'rms': np.float64(0.5826523379800569), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-...
FAILED tests/integration/test_contour_plot_examples.py::test_regression_npstere_da193_image - Failed: Image mismatch: {'rms': np.float64(0.011906453533254112), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-example-imag...
FAILED tests/integration/test_contour_plot_examples.py::test_example_7_lat_pressure_zonal_mean - Failed: Image mismatch for example 7: {'rms': np.float64(0.18799208017491612), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_8_log_scale_pressure - Failed: Image mismatch for example 8: {'rms': np.float64(0.11046718985438769), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_19_multiple_subplots - Failed: Image mismatch for example 19: {'rms': np.float64(4.15330387349586), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-e...
FAILED tests/integration/test_contour_plot_examples.py::test_example_21other_rgp_plasma - Failed: Image mismatch for example 21other: {'rms': np.float64(2.4241767024258962), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/refe...
FAILED tests/integration/test_contour_plot_examples.py::test_example_22_rgp_rotated_projection - Failed: Image mismatch for example 22: {'rms': np.float64(0.8955457536170187), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_23_rotated_grid_axes_overlay - Failed: Image mismatch for example 23: {'rms': np.float64(0.28493929640571825), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_34_lambert_conformal - Failed: Image mismatch for example 34: {'rms': np.float64(0.07193669150389208), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_35_mollweide_projection - Failed: Image mismatch for example 35: {'rms': np.float64(0.21729055554358212), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_36_mercator_projection - Failed: Image mismatch for example 36: {'rms': np.float64(0.22801506410925787), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_37_orthographic_projection - Failed: Image mismatch for example 37: {'rms': np.float64(0.19407875641259678), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_38_robinson_projection - Failed: Image mismatch for example 38: {'rms': np.float64(0.22239464014539573), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
======================================================= 24 failed, 72 passed, 3 deselected in 46.75s =======================================================
Steps to reproduce
Attempt to run tests in a different environment than those in which it was built.
Your environment
The version of cf-plot and its dependencies in which you are encountering an issue.
Platform: macOS-15.7.7-arm64-arm-64bit
Python: 3.12.13
packaging: 26.0
numpy: 2.4.3
cfdm.core: 1.13.0.0
udunits2 library: /Users/bnl28/miniforge3/envs/work26/bin/../lib/libudunits2.dylib
HDF5 library: 1.14.6
netcdf library: 4.9.3
netCDF4: 1.7.3
h5netcdf: 1.8.1
h5py: 3.16.0
pyfive: 1.2.0.dev79+g34ae13ead
zarr: 3.1.5
fsspec: 2026.2.0
scipy: 1.17.1
dask: 2026.3.0
distributed: 2026.3.0
cftime: 1.6.5
cfunits: 3.3.7
cfdm: 1.13.0.0
esmpy/ESMF: not available
psutil: 7.2.2
matplotlib: 3.10.8
activestorage: 0.4.0
cfplot: 3.5.0
cf: 3.19.0
Expected behaviour
Tests should pass
Actual behaviour
Too many fail, but when you investigate they're all trivial:
As anb eaxmple:
- 主要言語
- Python
- スター
- 21
- フォーク
- 11
- 平均マージ
- 3時間 36分
- マージ済み PR(30日)
- 2
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
NCAS-CMS/cf-plot のほかの issue
-
testing
難易度 3/5 1〜2日 初心者へのやさしさ 58/100
-
Add `AGENTS.md` fileオープンdocs enhancement
難易度 3/5 1〜2日 初心者へのやさしさ 64/100
-
Add animation hooks into cf-plot再び着手できるかも @bnlawrence が 103 日前に担当しましたが、オープン中のプルリクエストはありません。 オープンenhancement
-
bug v35plus
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
-
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
NCAS-CMS/cf-plot の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
spec-kitty/spec-kitty#6112 ·
メンテナーはふだん 1 日以内に返信
-
[Bug]: OpenRouter streaming chunks report the requested router name instead of the model OpenRouter served対応中かも @Mrun25 が今日担当しました。 オープンbug llm translation
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
BerriAI/litellm#46095 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
uberdeveloper/fastbt#29 ·
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
andrew-blake/melcloudhome#368 ·
メンテナーはふだん 1 日以内に返信
-
lane:discover
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
SPARDiffData/differential-data-collection#33 ·
メンテナーはふだん 1 日以内に返信