Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

dream and p_dream never check max_iterations against burn_in (default 10000), so a short run reports "Fitting complete" with no samples, histograms or credible intervals, while mh, pt and am refuse the same conf

Open Beginner friendly
  • Is something wrong?
#921 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Start in pybnf/algorithms/samplers/dream.py at DreamAlgorithm.start_run, then compare the existing checks in basic_mcmc.py and adaptive_mcmc.py. Reproduce the short-run configuration from the issue for dream and p_dream, and verify that invalid max_iterations and burn_in values are rejected before sampling rather than completing with empty output.

Written by the indexing model from the issue text.

Description

bug silent-incorrectness

The fix for #356 (reported by a user in #344) is incomplete. The max_iterations <= burn_in check it added is only in BasicBayesMCMCAlgorithm.start_run (pybnf/algorithms/samplers/basic_mcmc.py:118), which covers mh and pt. am has its own check (pybnf/algorithms/samplers/adaptive_mcmc.py:154). DreamAlgorithm inherits directly from BayesianAlgorithm (pybnf/algorithms/samplers/dream.py:97). Its start_run (dream.py:250) calls only BayesianAlgorithm.start_run, which does not check this, and PDreamAlgorithm (pybnf/algorithms/samplers/pdream.py:32) inherits it unchanged.

Every dream sampling site requires iteration > burn_in (dream.py:520, 670, 730, 845). So when max_iterations <= burn_in, no draw reaches samples.txt. This includes the case where burn_in is left at its default of 10000 (pybnf/algorithms/samplers/base.py:145). update_histograms then writes 'No samples collected — skipping histogram generation' to the log file only and returns (base.py:575-577), so no histograms or credible-interval files are written. The run exits 0 and prints R-hat/ESS lines followed by "Fitting complete". Nothing on the console says that sampling never started.

No number in the output is wrong. The best fit and sorted_params_final.txt are correct. The R-hat/ESS window includes draws from before burn_in, but that is documented (diagnostics_every in docs/config_keys.rst). The defect is that the posterior output is missing while the run reports success.

This has a side effect: cr_adapt_end = burn_in // 2 (dream.py:119) and p_dream's default precondition_adapt = burn_in // 2 (dream.py:248) both come out to 5000. In such a run, CR adaptation never freezes and p_dream never switches to its whitened proposal. In the reproduction below, p_dream's console output is byte-identical to dream's.

Failure scenario

A user runs job_type = dream or p_dream with max_iterations = 400 and no burn_in line. Burn-in is therefore 10000. All 400 iterations run, and the console shows convergence diagnostics and "Fitting complete". But Results/samples.txt contains only its header, Results/Histograms/ is empty, and there are no credible68/credible95 files. With job_type = mh, the same conf is refused before any simulation.

Reproduction

This uses the tutorial model and data in examples/tutorial/32_prior_gallery/ (bateman_chain.bngl, bateman_chain.exp). Put this dream.conf in the same directory:

edition = 2
model: bateman_chain.bngl
bngl_backend = bngsim
output_dir = out_dream

job_type = dream
noise_model = normal, sigma = read_exp_file _SD
experiment: timecourse, data: bateman_chain.exp

uniform_var = k1  0.05  3.0
uniform_var = k2  0.02  2.0

population_size = 4
max_iterations  = 400
sample_every    = 2
parallel_count  = 2
random_seed     = 1234

Run pybnf -c dream.conf (PyBNF v1.8.1).

Observed with job_type = dream:

  • Exit code 0. The console prints R-hat/ESS lines every 10 iterations. Its last lines are Max R-hat: 1.5542 (4 chains) / Min bulk ESS: 8.5 Min tail ESS: 5.9 / Stop criterion satisfied ... / Information criteria (best fit): ... / Fitting complete. No console line mentions samples or burn-in.
  • wc -l out_dream/Results/samples.txt gives 1 (header only). Results/Histograms/ is empty, and Results/ has no credible-interval files.
  • The only notice is in bnf_*.log: numpy's genfromtxt: Empty input file warning, then No samples collected — skipping histogram generation.

Observed with job_type = p_dream (and output_dir changed): the result is the same. Exit code 0, a header-only samples.txt, no histograms and no credible files. The console output is identical to dream's.

Expected, which is what job_type = mh does with the same conf: exit code 1 before any simulation. After mh's deprecation notice, it prints

Error: max_iterations (400) must be greater than burn_in (10000), otherwise no samples will be collected.

Reachability

Any dream or p_dream conf with max_iterations <= burn_in hits this. That includes every conf that omits burn_in and sets max_iterations to 10000 or less. None of the existing guards catches it:

  • the #356 check at basic_mcmc.py:118 runs only for mh/pt;
  • the check at adaptive_mcmc.py:154 runs only for am;
  • nothing in the config or parser layer compares burn_in with max_iterations (it only parses the key and supplies its default, base.py:145);
  • dream's own kalman_burnin_frac check (dream.py:198) validates only that fraction.

The obvious fix is to raise the same PybnfError from DreamAlgorithm.start_run.

Where

  • pybnf/algorithms/samplers/dream.py:250: start_run, no max_iterations/burn_in check
  • pybnf/algorithms/samplers/dream.py:520, 670, 730, 845: sampling requires iteration > burn_in
  • pybnf/algorithms/samplers/dream.py:119, 248: cr_adapt_end and precondition_adapt both default to burn_in // 2
  • pybnf/algorithms/samplers/pdream.py:32: inherits dream's start_run
  • pybnf/algorithms/samplers/basic_mcmc.py:118: the #356 check (mh/pt only)
  • pybnf/algorithms/samplers/adaptive_mcmc.py:154: am's equivalent check
  • pybnf/algorithms/samplers/base.py:145: burn_in default of 10000
  • pybnf/algorithms/samplers/base.py:575-577: the log-only 'No samples collected' warning

Related: #356, #862, #860.

Found in a whole-codebase audit for silently wrong results (2026-09-23); the reproduction above was re-run independently of the original finding.

Dominant language
Python
Stars
25
Forks
25
Avg merge
2h 38m
Merged PRs (30d)
98

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from lanl/PyBNF

All issues in lanl/PyBNF

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.