`bal_accuracy()`, `ppv()`, `npv()` return NA for macro averaging when a factor level is unused, unlike `j_index()`
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 85/100
Research direction
Start in R/class-bal_accuracy.R, R/class-ppv.R, and R/class-npv.R at the listed multiclass macro-average calculations, then compare them with j_index()'s handling of missing class estimates. Done means an unused factor level no longer makes the macro average NA, while the existing behavior for valid class-level estimates remains unchanged.
Written by the indexing model from the issue text.
Description
bal_accuracy_table_impl(), ppv_table_impl(), and npv_table_impl() all compute the multiclass macro average via stats::weighted.mean(out_vec, w) without na.rm = TRUE. If any class-level estimate is NA (e.g. an unused/empty factor level), the entire weighted mean becomes NA, even though sibling metrics such as j_index() handle this by passing na.rm = TRUE.
# R/class-bal_accuracy.R:139, R/class-ppv.R:197, R/class-npv.R:172
out_vec <- bal_accuracy_multiclass(data, estimator) # (or ppv_/npv_multiclass)
stats::weighted.mean(out_vec, w)
reprex:
library(yardstick)
truth <- factor(c("a", "a", "b", "b"), levels = c("a", "b", "c"))
estimate <- factor(c("a", "b", "a", "b"), levels = c("a", "b", "c"))
bal_accuracy_vec(truth, estimate)
#> Warning: While computing multiclass `recall()`, some levels had no true events (i.e.
#> `true_positive + false_negative = 0`).
#> Recall is undefined in this case, and those levels will be removed from the
#> averaged result.
#> Note that the following number of predicted events actually occurred for each
#> problematic event level:
#> 'c': 0
#> [1] NA
Created on 2026-08-31 with reprex v2.1.1
Suggested fix: add na.rm = TRUE to the three weighted.mean() calls, matching j_index()'s behavior.
- Dominant language
- R
- Stars
- 402
- Forks
- 63
- Avg merge
- 33m
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from tidymodels/yardstick
-
tidy-dev-day :nerd_face:
Difficulty 1/5 Under an hour Newbie friendliness 90/100
tidymodels/yardstick#637 ·
-
tidy-dev-day :nerd_face:
Difficulty 1/5 Under an hour Newbie friendliness 90/100
tidymodels/yardstick#635 ·
-
`poisson_log_loss()` overflows to Inf for large counts due to `log(gamma())` instead of `lgamma()` Opentidy-dev-day :nerd_face:
Difficulty 1/5 Under an hour Newbie friendliness 88/100
tidymodels/yardstick#634 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
tidymodels/yardstick#633 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
tidymodels/yardstick#629 ·
All issues in tidymodels/yardstick
Similar issues
-
documentation pkg infrastructure
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
epiverse-trace/epiparameter#511 ·
-
function:write_dwc
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
beginners blocker
Difficulty 2/5 1-3 hours Newbie friendliness 78/100