OLS random quantile sampling draws only ten residual values, shared across calls and variables

Aperta
#208 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
76/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
numpy, pandas, python

Direzione di ricerca

Start in microimpute/models/ols.py at OLSResults._predict_quantile and inspect the fixed sample count and generator initialization; compare its per-row drawing behavior with qrf.py:273. Use the reproduction in the issue to verify that predictions are no longer limited to ten values and that repeated calls and separate variables do not reuse the same draws.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

OLS._predict_quantile generates a fixed number of residual draws — ten — regardless of how many rows are being predicted, and rebuilds its generator from self.seed on every call. The imputed conditional distribution is therefore a ten-point discrete mixture, and repeated calls and separate target variables all receive the same ten values in the same row order.

Lower priority than #207 because this path is opt-in: random_quantile_sample defaults to False (ols.py:304), autoimpute() never reaches it (comparisons/autoimpute_helpers.py:373-377 always passes an explicit quantiles=[quantile]), and no caller inside the package sets it True. It is still wrong for anyone who does opt in.

Mechanism

# microimpute/models/ols.py:390
count_samples: int = 10
# microimpute/models/ols.py:426
rng = np.random.default_rng(self.seed)

Ten beta draws are mapped through norm.ppf and then assigned across all n test rows, so the realised moments of the imputed distribution are whatever those ten draws happened to be — not the fitted residual distribution.

Reproduction

import numpy as np, pandas as pd, logging
from microimpute.models.ols import OLSResults

o = OLSResults.__new__(OLSResults); o.seed = 42; o.logger = logging.getLogger("x")
v = o._predict_quantile(pd.Series(np.zeros(500)), 1.0, 0.5, True)
print(len(np.unique(v.values)))                                               # 10, not 500
print(np.allclose(v, o._predict_quantile(pd.Series(np.zeros(500)), 1.0, 0.5, True)))  # True

Ten distinct multipliers for 500 rows, and the second call is identical to the first.

Because the per-row standard error is proportional between two models fitted on the same X, the ratio of the two variables' shocks has exactly one distinct value across the test set — the two imputed variables are perfectly comonotonic. On data with a true residual correlation of −0.041, the imputed shocks correlate at 0.9999999999999999.

Suggested fix

Draw one variate per row (size=len(mean_preds)), and hold a single np.random.Generator on the results object, advanced across calls and variables, rather than reseeding from self.seed inside the method. qrf.py:273 already draws per row; OLS did not follow. The cross-variable half of the problem is the same root cause as #207.

Origin

Long-standing rather than a regression: eb5e88c, "quantile sampling for default imputation (quantreg and ols)", 7 May 2025. No comment suggests the count of ten is deliberate.

Found during a pre-JOSS-submission audit (#201), reproduced independently twice.

Lingua principale
Jupyter Notebook
Stelle
0
Fork
2
Merge medio
4g 23h
PR unite (30g)
5

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PolicyEngine/microimpute

Tutte le issue di PolicyEngine/microimpute

Issue simili

Altre issue su Data Engineering

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.