Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

dream and p_dream never check max_iterations against burn_in (default 10000), so a short run reports "Fitting complete" with no samples, histograms or credible intervals, while mh, pt and am refuse the same conf

Aperta Adatta ai principianti
  • Qualcosa non va?
#921 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
82/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend

Direzione di ricerca

Inizia in pybnf/algorithms/samplers/dream.py, in DreamAlgorithm.start_run, quindi confronta i controlli esistenti in basic_mcmc.py e adaptive_mcmc.py. Riproduci la configurazione di esecuzione breve dell’issue per dream e p_dream e verifica che i valori non validi di max_iterations e burn_in vengano rifiutati prima del campionamento, invece di completare con un output vuoto.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug silent-incorrectness

The fix for #356 (reported by a user in #344) is incomplete. The max_iterations <= burn_in check it added is only in BasicBayesMCMCAlgorithm.start_run (pybnf/algorithms/samplers/basic_mcmc.py:118), which covers mh and pt. am has its own check (pybnf/algorithms/samplers/adaptive_mcmc.py:154). DreamAlgorithm inherits directly from BayesianAlgorithm (pybnf/algorithms/samplers/dream.py:97). Its start_run (dream.py:250) calls only BayesianAlgorithm.start_run, which does not check this, and PDreamAlgorithm (pybnf/algorithms/samplers/pdream.py:32) inherits it unchanged.

Every dream sampling site requires iteration > burn_in (dream.py:520, 670, 730, 845). So when max_iterations <= burn_in, no draw reaches samples.txt. This includes the case where burn_in is left at its default of 10000 (pybnf/algorithms/samplers/base.py:145). update_histograms then writes 'No samples collected — skipping histogram generation' to the log file only and returns (base.py:575-577), so no histograms or credible-interval files are written. The run exits 0 and prints R-hat/ESS lines followed by "Fitting complete". Nothing on the console says that sampling never started.

No number in the output is wrong. The best fit and sorted_params_final.txt are correct. The R-hat/ESS window includes draws from before burn_in, but that is documented (diagnostics_every in docs/config_keys.rst). The defect is that the posterior output is missing while the run reports success.

This has a side effect: cr_adapt_end = burn_in // 2 (dream.py:119) and p_dream's default precondition_adapt = burn_in // 2 (dream.py:248) both come out to 5000. In such a run, CR adaptation never freezes and p_dream never switches to its whitened proposal. In the reproduction below, p_dream's console output is byte-identical to dream's.

Failure scenario

A user runs job_type = dream or p_dream with max_iterations = 400 and no burn_in line. Burn-in is therefore 10000. All 400 iterations run, and the console shows convergence diagnostics and "Fitting complete". But Results/samples.txt contains only its header, Results/Histograms/ is empty, and there are no credible68/credible95 files. With job_type = mh, the same conf is refused before any simulation.

Reproduction

This uses the tutorial model and data in examples/tutorial/32_prior_gallery/ (bateman_chain.bngl, bateman_chain.exp). Put this dream.conf in the same directory:

edition = 2
model: bateman_chain.bngl
bngl_backend = bngsim
output_dir = out_dream

job_type = dream
noise_model = normal, sigma = read_exp_file _SD
experiment: timecourse, data: bateman_chain.exp

uniform_var = k1  0.05  3.0
uniform_var = k2  0.02  2.0

population_size = 4
max_iterations  = 400
sample_every    = 2
parallel_count  = 2
random_seed     = 1234

Run pybnf -c dream.conf (PyBNF v1.8.1).

Observed with job_type = dream:

  • Exit code 0. The console prints R-hat/ESS lines every 10 iterations. Its last lines are Max R-hat: 1.5542 (4 chains) / Min bulk ESS: 8.5 Min tail ESS: 5.9 / Stop criterion satisfied ... / Information criteria (best fit): ... / Fitting complete. No console line mentions samples or burn-in.
  • wc -l out_dream/Results/samples.txt gives 1 (header only). Results/Histograms/ is empty, and Results/ has no credible-interval files.
  • The only notice is in bnf_*.log: numpy's genfromtxt: Empty input file warning, then No samples collected — skipping histogram generation.

Observed with job_type = p_dream (and output_dir changed): the result is the same. Exit code 0, a header-only samples.txt, no histograms and no credible files. The console output is identical to dream's.

Expected, which is what job_type = mh does with the same conf: exit code 1 before any simulation. After mh's deprecation notice, it prints

Error: max_iterations (400) must be greater than burn_in (10000), otherwise no samples will be collected.

Reachability

Any dream or p_dream conf with max_iterations <= burn_in hits this. That includes every conf that omits burn_in and sets max_iterations to 10000 or less. None of the existing guards catches it:

  • the #356 check at basic_mcmc.py:118 runs only for mh/pt;
  • the check at adaptive_mcmc.py:154 runs only for am;
  • nothing in the config or parser layer compares burn_in with max_iterations (it only parses the key and supplies its default, base.py:145);
  • dream's own kalman_burnin_frac check (dream.py:198) validates only that fraction.

The obvious fix is to raise the same PybnfError from DreamAlgorithm.start_run.

Where

  • pybnf/algorithms/samplers/dream.py:250: start_run, no max_iterations/burn_in check
  • pybnf/algorithms/samplers/dream.py:520, 670, 730, 845: sampling requires iteration > burn_in
  • pybnf/algorithms/samplers/dream.py:119, 248: cr_adapt_end and precondition_adapt both default to burn_in // 2
  • pybnf/algorithms/samplers/pdream.py:32: inherits dream's start_run
  • pybnf/algorithms/samplers/basic_mcmc.py:118: the #356 check (mh/pt only)
  • pybnf/algorithms/samplers/adaptive_mcmc.py:154: am's equivalent check
  • pybnf/algorithms/samplers/base.py:145: burn_in default of 10000
  • pybnf/algorithms/samplers/base.py:575-577: the log-only 'No samples collected' warning

Related: #356, #862, #860.

Found in a whole-codebase audit for silently wrong results (2026-09-23); the reproduction above was re-run independently of the original finding.

Lingua principale
Python
Stelle
25
Fork
25
Merge medio
10h 54m
PR unite (30g)
25

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di lanl/PyBNF

Tutte le issue di lanl/PyBNF

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.