run_simulation() is not reproducible from seed: the leisure draw uses dqrng's own RNG
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- r
- Domain
- data-engineering
Research direction
Read R/model.R and R/processes.R to trace how run_simulation() seeds the standard and dqrng generators, and check tests/testthat/test-processes.R for the existing reproducibility coverage. Implement the separate seeding fix from main, including the seed = NULL behavior and any necessary seed documentation. Done means two fresh runs with the same seed produce identical results without manually calling dqrng::dqset.seed().
Written by the indexing model from the issue text.
Description
Summary
Setting seed in the parameters does not make a run reproducible. The daily leisure redraw in create_SE_process() (R/processes.R) uses dqrng::dqsample.int(). dqrng has its own generator, which neither set.seed() nor parameters$seed touches. Two runs with the same seed therefore give different epidemics.
Reproducer
Checked in two fresh R processes, on main at 6c60e1b and on the ach_efficacy_update code (PR #142):
p <- get_parameters(list(human_population = 1500, number_initial_S = 1495,
number_initial_E = 5, simulation_time = 15, seed = 5))
run_simulation(p)$result$E_new # differs between fresh sessions
Calling dqrng::dqset.seed(5) before run_simulation() makes the output identical between runs.
Why it matters
- Anyone comparing scenarios with a fixed seed is comparing different leisure draws, so part of any difference between scenarios is noise.
- Regression tests on simulation output cannot be exact.
- It is easy to miss, because everything else is seeded.
Where the seeding happens today
run_simulation()callsset.seed(parameters_list$seed)inR/model.R.- The generators in
R/variables.Ralso callset.seed(parameters_list$seed). - dqrng is used in exactly one place, the leisure redraw in
R/processes.R(dqrng::dqsample.int(n = 7, ...)). That is the only unseeded generator.
Proposed fix
Call dqrng::dqset.seed() from parameters$seed in run_simulation(), next to the existing set.seed(). Points to settle in the fix:
- What to do when
seedisNULL, which is the default inget_parameters(). - Document that
seednow controls dqrng as well. - Add a test that two
run_simulation()calls with the sameseedgiveidentical()results without the caller touching dqrng.
Context
Found while profiling and vectorising the S→E process (branch perf/vectorize-se-process). That branch preserves the current behaviour exactly and does not fix this. Its equivalence tests call dqrng::dqset.seed() explicitly, so they keep working if run_simulation() seeds dqrng itself with the same value. The fix is meant to go in a separate small branch and PR, not in the performance branch.
Notes for whoever picks this up
- Work from
main. The one-line fix does not depend on the vectorisation branch. - On the vectorisation branch the leisure redraw is a single
dqsample.int(7, N, replace = TRUE)call that consumes the same stream as the earlier per-person calls, so the two changes do not interact. tests/testthat/test-processes.Ron that branch has a whole-run reproducibility test that seeds dqrng by hand. It can drop that call once this is fixed.
- Dominant language
- R
- Stars
- 8
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from petal-code/helios
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
petal-code/helios#174 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
petal-code/helios#173 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 84/100
petal-code/helios#166 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
petal-code/helios#161 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
petal-code/helios#175 ·
All issues in petal-code/helios
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
atomashevic/transforEmotion#46 ·
Maintainers usually reply within 1 day
-
Error when applying yearCohorts() to a cohort created with generateTargetDenominatorCohortSet()Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
OHDSI/CohortConstructor#777 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
nmfs-ost/stockplotr#445 · 2 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
Use renv::installOpen
Difficulty 1/5 Under an hour Newbie friendliness 72/100
UCL-ARC/r-intermediate-good-practices#37 ·
Maintainers usually reply within 2 days