Track all of our RNG offsets to avoid collisions

Description

(base) √ metaseq % ag seed --py | grep +
cpu_tests/test_streaming_token_block_dataset.py:78:        shadow_rng = np.random.default_rng(2273 + seed)
cpu_tests/test_streaming_token_block_dataset.py:124:        shadow_rng = np.random.default_rng(2273 + seed)
metaseq/tasks/language_modeling.py:217:            with data_utils.numpy_seed(self.args.seed + epoch):
metaseq/tasks/streaming_language_modeling.py:316:            seed=1284 + self.args.seed,
metaseq/trainer.py:1052:        seed = self.cfg.common.seed + self.get_num_updates()
metaseq/data/streaming_token_block_dataset.py:96:            rng = np.random.default_rng(2273 + self.seed)
metaseq/data/iterators.py:524:                batches = shuffle_batches(list(batches), self.seed + epoch)
metaseq/data/iterators.py:532:                batches = shuffle_batches(batches, self.seed + epoch + self.shard_id)
metaseq/data/iterators.py:535:                batches = shuffle_batches(list(self.frozen_batches), self.seed + epoch)

Contributor guide

Tech stack

pythonnumpy

Domain

machine learningdata

Issue type

feature

DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.

3

Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.

half day

Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.

stale

ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.

clear

Prerequisites

PythonNumPy

Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.

50

Research direction

First, use grep or similar to find all RNG seed offsets in the codebase, building on the examples provided. Then, design a centralized registry (e.g., a dictionary or config) to store and avoid collisions. Review files such as cpu tests/test streaming token block dataset.py, metaseq/tasks/language modeling.py, and metaseq/trainer.py to understand how offsets are currently used. Finally, propose a pull request that implements the tracking mechanism.