Trainer reuses the same data permutation every epoch
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning
Research direction
Locate the trainer code containing the shown epoch loop and inspect how JAX PRNG keys are passed to iterate_dataset. Ensure training receives a freshly split key for each epoch, then verify that successive epochs no longer reuse the same data permutation.
Written by the indexing model from the issue text.
Description
The trainer creates the shuffle key once, outside the epoch loop:
key, subkey1, subkey2 = jax.random.split(key, 3)
for epoch in range(train_epochs + 1):
for x, y in iterate_dataset(
train_ds,
nbatches,
batch_size,
subkey1,
True,
):
Since JAX PRNG keys are immutable, subkey1 produces the same permutation every epoch.
Expected behavior is to generate a fresh training key each epoch, e.g.
for epoch in range(train_epochs):
key, train_key = jax.random.split(key)
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lanl/RMC
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100