nwss_hhs_region aggregation computes `region_value = mean(region_value * population) / sum(population)` — off by a factor of the region's state count, and NA-prone
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- r
- Domain
- data-engineering
Research direction
Start in R/targets/flu_data_targets.R around line 194 and compare the nwss_hhs_region aggregation with its sibling statistics; then inspect the identical block in R/targets/covid_data_targets.R around line 135. Use the two-state population-weighted repro from the issue and verify both paths handle missing populations without producing an unintended NA. Done means the regional value matches the intended sum/sum calculation in both files.
Written by the indexing model from the issue text.
Description
Severity (my assessment): medium
Area: data targets (R/targets/flu_data_targets.R, R/targets/covid_data_targets.R)
Commit observed: current main (ccc2ee4); also verified at 5e35f53
Related existing issue: #165 (open, "Double-check NWSS national aggregations" — overlapping concern about the NWSS aggregation paths; my report is a distinct, specific defect on the regional aggregation)
Summary
In the nwss_hhs_region aggregation, four sibling statistics use a population-weighted sum/sum, but region_value uses mean/sum:
R/targets/flu_data_targets.R:194 (identical at R/targets/covid_data_targets.R:135):
summarize(
value = sum(value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
activity_level = sum(activity_level * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
region_value = mean(region_value * population) / sum(population, na.rm = TRUE),
national_value = sum(national_value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
...
)
mean(region_value * population) is the per-state average of the weighted numerator — off by a factor of the region's state count (e.g. ÷9 for a 9-state region... effectively multiplied by 1/9 relative to the intended sum). Also, the bare mean() (no na.rm) returns NA if any population in the group is NA, so a single missing population poisons that region's row entirely.
Maintainer-runnable repro
sed -n '188,199p' R/targets/flu_data_targets.R
sed -n '130,140p' R/targets/covid_data_targets.R
Or in R: build a two-state group with populations 1M and 3M and equal region_value — the current formula returns region_value * 2M / 4M-scaled arithmetic that diverges from the intended sum/sum by the state count.
Impact
Miscalibrated nwss_region exogenous predictor in explore sweeps. As far as I can tell prod is unaffected here (the prod path drops nwss rows upstream via drop_na(hhs)), so this is an explore/report-path correctness issue.
Suggested fix
region_value = sum(region_value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
in both files (flu + covid), matching the sibling statistics.
- Dominant language
- HTML
- Stars
- 2
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from cmu-delphi/exploration-tooling
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
All issues in cmu-delphi/exploration-tooling
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
open-compass/VLMEvalKit#1698 ·
-
triage:deciding
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
open-telemetry/otel-arrow#4123 · 1 reaction ·