Tests that run on CRAN, possibly missing changes in other packages

未關閉
  • 哪裡不對？
#415 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 1 天內回覆

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
48/100
Issue 類型
重構
描述清晰度
基本清楚
活躍度
活躍
技術堆疊
r
領域
testing-qa

研究方向

Start by reviewing the snapshot-only test blocks and the five cores = 2 blocks named in the issue, including test_psis.R, test_gpdfit.R, test_relative_eff.R, test_E_loo.R, and test_loo_and_waic.R. Check testthat’s CRAN snapshot behavior and the CRAN macOS restriction described in the issue; decide which tests should run on CRAN and add fixed-value checks or narrower skips accordingly. Done means the relevant posterior numeric behavior and parallel paths are exercised on CRAN without exceeding its stated limits.

由索引模型根據 Issue 內容生成。

描述

projpred encountered a problem where some slow tests depending on other packages were not run on CRAN, and lme4 made a change which was then not caught by reverse dependency checks. I checked loo tests and currently there are also some things that are not tested in CRAN. The good thing is that those other packages are rstan, rstanarm, and posterior, so it is likely we would know about the changes.

Snapshot tests do not run on CRAN.

In testthat 3.3.2, expect_snapshot() and expect_snapshot_value() default to cran = FALSE, and loo never sets cran = TRUE. This is the biggest gap, larger than the skip_on_cran() blocks. Thirteen test_that() blocks contain only snapshot expectations, so on CRAN they check nothing. These blocks guard the numbers that loo gets from posterior:

loo code posterior function fixed-value check that runs on CRAN
psis() main path gpdfit(sort_x = FALSE) none (test_psis.R:13 is a snapshot)
psis() smoothing qgeneralized_pareto() none
loo::gpdfit() gpdfit(wip, min_grid_pts) none (test_gpdfit.R is snapshots only)
psislw() (old) gpdfit(wip = FALSE), qgp yes, test_psislw.R:30-33
relative_eff() ess_mean() none (test_relative_eff.R:7); line 14 only compares against ess_mean
E_loo() diagnostic pareto_khat(tail, ndraws_tail) none (test_E_loo.R:136,144)
loo()/waic() all the above plus matrixStats none (test_loo_and_waic.R:23)
  • Interface changes are already caught on CRAN. These functions run in tests that CRAN does run, so a removed or renamed posterior argument would cause an error.
  • Numeric changes are not caught. Examples are a new default grid, prior or tail length in gpdfit, ess_mean or pareto_khat. Only the old psislw() path has hard-coded values.

Should we

  • enable some snapshot tests in CRAN?
  • add new guard tests with small fixed non-random inputs that are run on CRAN?
  • trust that when something changes in posterior, we remember to check things in loo?
parallel tests do not run on CRAN

None of the cores = 2 code runs on CRAN. That covers fork (mclapply) and PSOCK (parLapply, plus clusterExport in relative_eff). The stated reason applies only to macOS.

Should we

  • in the 5 cores = 2 blocks, skip only on CRAN macOS, instead of on all of CRAN? This uses at most 2 cores, which CRAN allows.
主要語言
R
星號
159
分支
37
平均合併
18 小時 1 分鐘
30 天內合併 PR
8

環境準備

  • 沒有 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

stan-dev/loo 的其他 Issue

查看 stan-dev/loo 的全部 Issue

相似的 Issue

更多 R Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。