Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Testing is far too fragile

Aperta
#135 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
22/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
matplotlib, python
Ambito
testing

Direzione di ricerca

The failures come from the image comparison in tests/integration/test_advanced_plot_examples.py and tests/integration/test_contour_plot_examples.py, which compare rendered plots against reference images in tests/reference-example-images. Run one failing case, such as test_example_1_basic_cylindrical, and read how the rms value is compared with the reference. The issue does not say whether the fix should be a tolerance change, regenerated references, or environment pinning, so a maintainer decision is needed first. Done means the suite passes across environments under the chosen approach.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Subject of the issue

I've just pulled the latest release, and 24 tests fail. They are all minor fails to do with image mismatch, but it's just too fragile.

================================================================= short test summary info ==================================================================
FAILED tests/integration/test_advanced_plot_examples.py::test_example_14_vector_with_colour_contour - Failed: Image mismatch for example 14: {'rms': np.float64(0.07727821087173495), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_15_polar_vector_plot - Failed: Image mismatch for example 15: {'rms': np.float64(1.2350842133667295), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_16b_basic_stream_plot - Failed: Image mismatch for example 16b: {'rms': np.float64(19.617677264399287), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_16c_enhanced_stream_plot - Failed: Image mismatch for example 16c: {'rms': np.float64(21.34025810212168), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_17_basic_stipple_plot - Failed: Image mismatch for example 17: {'rms': np.float64(0.21548615766477328), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_18_polar_stipple_plot - Failed: Image mismatch for example 18: {'rms': np.float64(0.14397116764560633), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_advanced_plot_examples.py::test_canari_1 - Failed: Image mismatch for example canari_1: {'rms': np.float64(0.02305860896416126), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/re...
FAILED tests/integration/test_advanced_plot_examples.py::test_example_40_tracks_polar_stereographic - Failed: Image mismatch for example 40: {'rms': np.float64(0.01662284425174423), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_1_basic_cylindrical - Failed: Image mismatch for example 1: {'rms': np.float64(0.18352397386241887), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_3_map_limits_and_levels - Failed: Image mismatch for example 3: {'rms': np.float64(0.08961256746282698), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_4_north_pole_stereographic - Failed: Image mismatch for example 4: {'rms': np.float64(0.5533641260245302), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-...
FAILED tests/integration/test_contour_plot_examples.py::test_example_5_south_pole_with_boundary - Failed: Image mismatch for example 5: {'rms': np.float64(0.5826523379800569), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-...
FAILED tests/integration/test_contour_plot_examples.py::test_regression_npstere_da193_image - Failed: Image mismatch: {'rms': np.float64(0.011906453533254112), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-example-imag...
FAILED tests/integration/test_contour_plot_examples.py::test_example_7_lat_pressure_zonal_mean - Failed: Image mismatch for example 7: {'rms': np.float64(0.18799208017491612), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_8_log_scale_pressure - Failed: Image mismatch for example 8: {'rms': np.float64(0.11046718985438769), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_19_multiple_subplots - Failed: Image mismatch for example 19: {'rms': np.float64(4.15330387349586), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference-e...
FAILED tests/integration/test_contour_plot_examples.py::test_example_21other_rgp_plasma - Failed: Image mismatch for example 21other: {'rms': np.float64(2.4241767024258962), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/refe...
FAILED tests/integration/test_contour_plot_examples.py::test_example_22_rgp_rotated_projection - Failed: Image mismatch for example 22: {'rms': np.float64(0.8955457536170187), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/reference...
FAILED tests/integration/test_contour_plot_examples.py::test_example_23_rotated_grid_axes_overlay - Failed: Image mismatch for example 23: {'rms': np.float64(0.28493929640571825), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_34_lambert_conformal - Failed: Image mismatch for example 34: {'rms': np.float64(0.07193669150389208), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_35_mollweide_projection - Failed: Image mismatch for example 35: {'rms': np.float64(0.21729055554358212), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_36_mercator_projection - Failed: Image mismatch for example 36: {'rms': np.float64(0.22801506410925787), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_37_orthographic_projection - Failed: Image mismatch for example 37: {'rms': np.float64(0.19407875641259678), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
FAILED tests/integration/test_contour_plot_examples.py::test_example_38_robinson_projection - Failed: Image mismatch for example 38: {'rms': np.float64(0.22239464014539573), 'expected': '/Users/bnl28/syncthing/Repositories/cf-plot/tests/referenc...
======================================================= 24 failed, 72 passed, 3 deselected in 46.75s =======================================================
Steps to reproduce

Attempt to run tests in a different environment than those in which it was built.

Your environment

The version of cf-plot and its dependencies in which you are encountering an issue.

Platform: macOS-15.7.7-arm64-arm-64bit
Python: 3.12.13
packaging: 26.0
numpy: 2.4.3
cfdm.core: 1.13.0.0
udunits2 library: /Users/bnl28/miniforge3/envs/work26/bin/../lib/libudunits2.dylib
HDF5 library: 1.14.6
netcdf library: 4.9.3
netCDF4: 1.7.3
h5netcdf: 1.8.1
h5py: 3.16.0
pyfive: 1.2.0.dev79+g34ae13ead
zarr: 3.1.5
fsspec: 2026.2.0
scipy: 1.17.1
dask: 2026.3.0
distributed: 2026.3.0
cftime: 1.6.5
cfunits: 3.3.7
cfdm: 1.13.0.0
esmpy/ESMF: not available
psutil: 7.2.2
matplotlib: 3.10.8
activestorage: 0.4.0
cfplot: 3.5.0
cf: 3.19.0
Expected behaviour

Tests should pass

Actual behaviour

Too many fail, but when you investigate they're all trivial:

As anb eaxmple:
Image
Image

Lingua principale
Python
Stelle
21
Fork
11
Merge medio
3h 36m
PR unite (30g)
2

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NCAS-CMS/cf-plot

Tutte le issue di NCAS-CMS/cf-plot

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.