nwss_hhs_region aggregation computes `region_value = mean(region_value * population) / sum(population)` — off by a factor of the region's state count, and NA-prone

Open Beginner friendly
#226 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
86/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
r

Research direction

Start in R/targets/flu_data_targets.R around line 194 and compare the nwss_hhs_region aggregation with its sibling statistics; then inspect the identical block in R/targets/covid_data_targets.R around line 135. Use the two-state population-weighted repro from the issue and verify both paths handle missing populations without producing an unintended NA. Done means the regional value matches the intended sum/sum calculation in both files.

Written by the indexing model from the issue text.

Description

Severity (my assessment): medium
Area: data targets (R/targets/flu_data_targets.R, R/targets/covid_data_targets.R)
Commit observed: current main (ccc2ee4); also verified at 5e35f53
Related existing issue: #165 (open, "Double-check NWSS national aggregations" — overlapping concern about the NWSS aggregation paths; my report is a distinct, specific defect on the regional aggregation)

Summary

In the nwss_hhs_region aggregation, four sibling statistics use a population-weighted sum/sum, but region_value uses mean/sum:

R/targets/flu_data_targets.R:194 (identical at R/targets/covid_data_targets.R:135):

summarize(
  value = sum(value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
  activity_level = sum(activity_level * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
  region_value = mean(region_value * population) / sum(population, na.rm = TRUE),
  national_value = sum(national_value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),
  ...
)

mean(region_value * population) is the per-state average of the weighted numerator — off by a factor of the region's state count (e.g. ÷9 for a 9-state region... effectively multiplied by 1/9 relative to the intended sum). Also, the bare mean() (no na.rm) returns NA if any population in the group is NA, so a single missing population poisons that region's row entirely.

Maintainer-runnable repro

sed -n '188,199p' R/targets/flu_data_targets.R
sed -n '130,140p' R/targets/covid_data_targets.R

Or in R: build a two-state group with populations 1M and 3M and equal region_value — the current formula returns region_value * 2M / 4M-scaled arithmetic that diverges from the intended sum/sum by the state count.

Impact

Miscalibrated nwss_region exogenous predictor in explore sweeps. As far as I can tell prod is unaffected here (the prod path drops nwss rows upstream via drop_na(hhs)), so this is an explore/report-path correctness issue.

Suggested fix

region_value = sum(region_value * population, na.rm = TRUE) / sum(population, na.rm = TRUE),

in both files (flu + covid), matching the sibling statistics.

Dominant language
HTML
Stars
2
Forks
3
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cmu-delphi/exploration-tooling

All issues in cmu-delphi/exploration-tooling

Similar issues

More Data Engineering issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.