An experiment name equal to another experiment's name plus a condition name collides in both exp_data and the simulation dict: one dataset is dropped and the other is scored against the wrong simulation
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 65/100
Research direction
The bug is in pybnf/config.py line 1979 and pybnf/pset.py line 986. Start by reading the issue's reproduction steps and the linked code. Run the provided example with correct.conf and coll.conf to see the dataset collision. Check how experiment names and condition names are concatenated to form keys in exp_data and simulation dicts. Verify the fix by ensuring unique keys and that no data is silently dropped.
Written by the indexing model from the issue text.
Description
A mutant's outputs are keyed by plain string concatenation, ds[suff + mut.suffix] (BNGLModel.execute 986; also SbmlModelNoTimeout 1640/1690 and the bngsim executes). These keys overwrite a base action whose suffix happens to equal the concatenation. On the config side, _resolve_experiment_data_key returns name+condition (config.py:1979) with no check against other experiment names, so exp_data[base][key] is also overwritten. The legacy analogue is action suffixes 'a' and 'ab' plus mutant 'b'.
Failure scenario
inducible_gene.bngl with 'condition: off (Stimulus_isOn = 0)', 'experiment: rise, condition: off, data: flat.exp', and 'experiment: riseoff, data: rise.exp'. The fit completes with n=6 (one 6-point dataset silently dropped). rise.exp is scored against the 'off' mutant of 'rise' (all zeros), so the objective is flat in k_deg, and the fit reports k_deg=2.75 from generation 0 (truth 2).
Reproduction (independent re-run)
pybnf loaded from pybnf/init.py.
LOAD (load_config):
- correct.conf, where the same two datasets are named wildtype and knockout+ko: get_suffixes ['wildtype','wildtypeko','knockout','knockoutko'], exp_data keys ['knockoutko','wildtype'], 34 data rows in total.
- coll.conf: get_suffixes ['wt','wtko','wtko','wtkoko'], exp_data keys ['wtko'] only, holding the wildtype data (Obs_B[-1]=62.18). experiment data keys {'wt': ('reversible_conversion','wtko'), 'wtko': ('reversible_conversion','wtko')}, 17 data rows in total. The knockout dataset is gone.
FIT (de, pop 24, seed 7, refine):
- correct.conf: 'objective function value of 1.08e-13', 'n=34', best kf=0.70000000, kr=0.19999999. The truth is 0.7/0.2.
- coll.conf: rc=0, 'Fitting complete', 'objective function value of 680.0964593695738', 'n=17', best kf=0.3707, kr=1.5173. kr is unidentified (1.5115 at the same objective). No warning in the console or the bnf_*.log, and FailedSimLogs is empty.
INDEPENDENT CHECK (numpy/scipy, analytic solutions):
- chi_sq of the wildtype data against the knockout curve at PyBNF's best: 680.0965 (matches PyBNF).
- The same data against the wildtype curve: 13711.1.
- The independent minimum over kf of wildtype data against the knockout curve: kf=0.37070, obj 680.0965.
- At the truth, both datasets fit with chi_sq about 1e-11.
Reachability
Any edition-2 conf in which an experiment name plus its condition name equals another experiment's name. I reproduced it with the Lesson-47 model (examples/tutorial/47_condition_perturbations/reversible_conversion.bngl) and this conf: edition = 2, model: reversible_conversion.bngl, bngl_backend = bngsim, condition: ko, perturbations: kr = 0, experiment: wt, condition: ko, data: wt.exp, experiment: wtko, data: wtko.exp, with job_type = de and objective = chi_sq. Every key used is documented. Two name pairs whose concatenations coincide ('a'+'bc' and 'ab'+'c') also trigger it. The trigger is a naming coincidence, but 'wt'/'ko'/'wtko'-style names are natural in biology, and nothing rejects them.
Where
pybnf/config.py:1979 pybnf/pset.py:986
Found in a whole-codebase audit for silently wrong results (2026-09-23); the reproduction above was re-run independently of the original finding.
- Dominant language
- Python
- Stars
- 25
- Forks
- 25
- Avg merge
- 2h 26m
- Merged PRs (30d)
- 83
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lanl/PyBNF
-
documentation
Difficulty 1/5 Under an hour Newbie friendliness 91/100
-
bug silent-incorrectness
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
bug silent-incorrectness
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug silent-incorrectness
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
bug silent-incorrectness
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
stephrobert/dsoxlab#238 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
sublimehq/package_control#1780 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
nwg-piotr/nwg-displays#145 ·