OLS random quantile sampling draws only ten residual values, shared across calls and variables
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 76/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- numpy, pandas, python
- Ambito
- data, machine-learning
Direzione di ricerca
Start in microimpute/models/ols.py at OLSResults._predict_quantile and inspect the fixed sample count and generator initialization; compare its per-row drawing behavior with qrf.py:273. Use the reproduction in the issue to verify that predictions are no longer limited to ten values and that repeated calls and separate variables do not reuse the same draws.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
OLS._predict_quantile generates a fixed number of residual draws — ten — regardless of how many rows are being predicted, and rebuilds its generator from self.seed on every call. The imputed conditional distribution is therefore a ten-point discrete mixture, and repeated calls and separate target variables all receive the same ten values in the same row order.
Lower priority than #207 because this path is opt-in: random_quantile_sample defaults to False (ols.py:304), autoimpute() never reaches it (comparisons/autoimpute_helpers.py:373-377 always passes an explicit quantiles=[quantile]), and no caller inside the package sets it True. It is still wrong for anyone who does opt in.
Mechanism
# microimpute/models/ols.py:390
count_samples: int = 10
# microimpute/models/ols.py:426
rng = np.random.default_rng(self.seed)
Ten beta draws are mapped through norm.ppf and then assigned across all n test rows, so the realised moments of the imputed distribution are whatever those ten draws happened to be — not the fitted residual distribution.
Reproduction
import numpy as np, pandas as pd, logging
from microimpute.models.ols import OLSResults
o = OLSResults.__new__(OLSResults); o.seed = 42; o.logger = logging.getLogger("x")
v = o._predict_quantile(pd.Series(np.zeros(500)), 1.0, 0.5, True)
print(len(np.unique(v.values))) # 10, not 500
print(np.allclose(v, o._predict_quantile(pd.Series(np.zeros(500)), 1.0, 0.5, True))) # True
Ten distinct multipliers for 500 rows, and the second call is identical to the first.
Because the per-row standard error is proportional between two models fitted on the same X, the ratio of the two variables' shocks has exactly one distinct value across the test set — the two imputed variables are perfectly comonotonic. On data with a true residual correlation of −0.041, the imputed shocks correlate at 0.9999999999999999.
Suggested fix
Draw one variate per row (size=len(mean_preds)), and hold a single np.random.Generator on the results object, advanced across calls and variables, rather than reseeding from self.seed inside the method. qrf.py:273 already draws per row; OLS did not follow. The cross-variable half of the problem is the same root cause as #207.
Origin
Long-standing rather than a regression: eb5e88c, "quantile sampling for default imputation (quantreg and ols)", 7 May 2025. No comment suggests the count of ten is deliberate.
Found during a pre-JOSS-submission audit (#201), reproduced independently twice.
- Lingua principale
- Jupyter Notebook
- Stelle
- 0
- Fork
- 2
- Merge medio
- 4g 23h
- PR unite (30g)
- 5
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di PolicyEngine/microimpute
-
Pre-submission cleanup: API exports, validation, CI gaps, vacuous tests, dead code, stale docs Aperta
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
PolicyEngine/microimpute#213 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 52/100
PolicyEngine/microimpute#212 ·
-
Log loss is computed from class labels, and integer counts are silently switched to classification Aperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
PolicyEngine/microimpute#209 ·
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
PolicyEngine/microimpute#206 ·
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 55/100
PolicyEngine/microimpute#205 ·
Tutte le issue di PolicyEngine/microimpute
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
cisagov/cyhy-reports#149 · 3 commenti ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
open-compass/VLMEvalKit#1698 ·