loo_model_weights() rejects PSIS-LOO objects with equal observations but different posterior sample sizes
还没有人认领这个 Issue。
评估
调研方向
从 loo_model_weights() 和 validate_psis_loo_list() 开始,然后将它们的维度检查与 loo_compare() 进行比较。使用 loo_full 和 loo_short 重现该示例,并为观测数量相同但后验样本大小不同的情况添加覆盖。完成的标准是 loo_model_weights() 接受这些对象,同时保留对不兼容观测维度的验证。
由索引模型根据 Issue 内容生成。
描述
Both objects contain pointwise LOO results for the same 32 observations in the same order. The only difference is the number of posterior draws: 1,000 versus 500 after merging chains.
loo_compare() accepts these objects, but loo_model_weights() rejects them because validate_psis_loo_list() requires both dimensions of the psis_loo objects to match.
My understanding is that stacking and pseudo-BMA operate on the pointwise elpd_loo vectors, so the posterior sample sizes need not be identical. Different sample sizes affect Monte Carlo precision but should not make the objects incompatible.
example
library(loo)
log_lik_full <- example_loglik_array()
# Keep the same chains and 32 observations, but use half the iterations.
log_lik_short <- log_lik_full[seq_len(dim(log_lik_full)[1] / 2),,, drop = FALSE]
loo_full <- loo(
log_lik_full,
r_eff = relative_eff(exp(log_lik_full))
)
loo_short <- loo(
log_lik_short,
r_eff = relative_eff(exp(log_lik_short))
)
dim(loo_full)
#> [1] 1000 32
dim(loo_short)
#> [1] 500 32
# Comparison works despite the different posterior sample sizes.
loo_compare(loo_full, loo_short)
# Model weights do not.
loo_model_weights(list(
full = loo_full,
short = loo_short
))
sessionInfo()
output
> library(loo)
This is loo version 2.10.1
- Online documentation and vignettes at mc-stan.org/loo
- As of v2.0.0 loo defaults to 1 core but we recommend using as many as possible. Use the 'cores' argument or set options(mc.cores = NUM_CORES) for an entire session.
- Windows 10 users: loo may be very slow if 'mc.cores' is set in your .Rprofile file (see https://github.com/stan-dev/loo/issues/94).
Warning message:
package ‘loo’ was built under R version 4.6.1
>
> log_lik_full <- example_loglik_array()
>
> # Keep the same chains and 32 observations, but use half the iterations.
> log_lik_short <- log_lik_full[seq_len(dim(log_lik_full)[1] / 2),,, drop = FALSE]
>
> loo_full <- loo(
+ log_lik_full,
+ r_eff = relative_eff(exp(log_lik_full))
+ )
>
> loo_short <- loo(
+ log_lik_short,
+ r_eff = relative_eff(exp(log_lik_short))
+ )
>
> dim(loo_full)
[1] 1000 32
> #> [1] 1000 32
>
> dim(loo_short)
[1] 500 32
> #> [1] 500 32
>
> # Comparison works despite the different posterior sample sizes.
> loo_compare(loo_full, loo_short)
model elpd_diff se_diff p_worse diag_diff diag_elpd
model2 0.0 0.0 NA
model1 0.0 0.1 0.70 N < 100
Diagnostic flags present.
See ?`loo-glossary` (sections `diag_diff` and `diag_elpd`)
or https://mc-stan.org/loo/reference/loo-glossary.html.
>
> # Model weights do not.
> loo_model_weights(list(
+ full = loo_full,
+ short = loo_short
+ ))
Error: Each object in the list must have the same dimensions.
> sessionInfo()
R version 4.6.0 (2026-04-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)
Matrix products: default
LAPACK version 3.12.1
locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
time zone: Europe/Amsterdam
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] loo_2.10.1
loaded via a namespace (and not attached):
[1] vctrs_0.7.3 cli_3.6.6 knitr_1.51 rlang_1.2.0
[5] xfun_0.57 otel_0.2.0 generics_0.1.4 tensorA_0.36.2.1
[9] glue_1.8.1 backports_1.5.1 htmltools_0.5.9 distributional_0.7.0
[13] rmarkdown_2.31 evaluate_1.0.5 tibble_3.3.1 abind_1.4-8
[17] fastmap_1.2.0 yaml_2.3.12 lifecycle_1.0.5 compiler_4.6.0
[21] posterior_1.7.0 pkgconfig_2.0.3 rstudioapi_0.18.0 digest_0.6.39
[25] pillar_1.11.1 parallel_4.6.0 magrittr_2.0.5 checkmate_2.3.4
[29] tools_4.6.0 matrixStats_1.5.0
- 主要语言
- R
- 星标
- 157
- 派生
- 38
- 平均合并
- 4 天 16 小时
- 30 天内合并 PR
- 2
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
stan-dev/loo 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 5/5 一周以上 新手友好度 25/100
-
难度 3/5 1-2 天 新手友好度 45/100
-
Clean up issues 未关闭
难度 4/5 3-5 天 新手友好度 25/100
-
难度 3/5 1-2 天 新手友好度 68/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 75/100
easystats/performance#950 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 2/5 1-3 小时 新手友好度 75/100
briandconnelly/airnow#9 ·
-
bug
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 2/5 1-3 小时 新手友好度 75/100
OHDSI/CohortConstructor#774 ·