Tests: order-dependent snapshots, unseeded data, silent skips and assertions that cannot fail

Aperta
  • Qualcosa non va?
#1,275 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Refactoring
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
r
Ambito
testing

Direzione di ricerca

Start with test-impute-missing-scores.R, test-metrics-binary.R, test-class-forecast-ordinal.R, and the print snapshot tests, then inspect shared fixtures in setup.R. Run the review's filtered test commands to reproduce the order dependent snapshots and a full test run to check random failures, skips, assertions, prints, warnings, and labels. Done when these test issues are addressed and the relevant tests pass without unintended skips or warnings.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

CLAUDE: - [ ] Six plot snapshots depend on test order. test-impute-missing-scores.R does scores <- scores_quantile and later data.table::setkeyv(scores, names(scores)), which sorts the shared fixture from setup.R by reference. The vdiffr snapshots of plot_heatmap(), plot_wis() and plot_pairwise_comparisons() were recorded against the sorted fixture:

devtools::test(filter = "plot_heatmap|plot_wis")                        # 4 failures
devtools::test(filter = "impute-missing-scores|plot_heatmap|plot_wis")  # 0 failures

Fix: data.table::copy() in that test and re-record the snapshots.

  • test-metrics-binary.R draws observed with rbinom() and no seed at file level; when all ten draws are equal (about 1 in 400) six tests fail.
  • Eleven skip_if_not() guards in test-impute-missing-scores.R (e.g. on exists("build_missing_grid", ...)) would silently skip tests of exported functions if an internal helper were renamed.
  • Assertions that cannot fail: expect_s3_class(x, c("forecast_quantile", "data.table", "data.frame")) passes for any data.frame (no exact = TRUE); seven bare expect_error() / expect_warning() calls without a message; get_metrics() tests only check expect_type(, "list"); the permutation test only uses expect_no_condition(); get_pit_histogram() tests never check a density.
  • Print tests: nominal, ordinal and multivariate are not in the print snapshot test, and the same print() is snapshotted four times per data set.
  • Unused fixtures in setup.R (scores_sample_discrete, scores_nominal, metrics_no_cov_no_ae); a clean run has four warnings that nothing fails on.
  • Mislabelled tests in test-class-forecast-ordinal.R ("get_metrics.forecast_nominal()", "is_forecast_nominal() works").

Related: #685, #686, #986.

Found during a full review of the package (version 2.3.0.9000, main at 779d7d68).

Lingua principale
R
Stelle
65
Fork
24
Merge medio
1h 24m
PR unite (30g)
2

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di epiforecasts/scoringutils

Tutte le issue di epiforecasts/scoringutils

Issue simili

Altre issue su R

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.