Training only processes one trajectory at a time?
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 42/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start with src/art/preprocessing/inputs.py at the referenced line and trace how trajectories become training sequences. Compare that path with the per_device_train_batch_size configuration using the reported training loop and logs. Done means determining whether processing one sample at a time is intentional and, if not, making the configured batching behavior work or documenting the rationale.
Written by the indexing model from the issue text.
Description
I'm running the training loop with this code, and noticed the trainer only processes one trajectory at a time.
for step in range(TRAINING_STEPS):
print(f"Step {step+1} rollout")
train_groups = []
for scenario in scenarios:
trajectories = await rollout(model.name, vllm_url, scenario)
rewards = [traj.reward for traj in trajectories]
print(f"\n{scenario['id']} rewards: {rewards}")
train_groups.append(art.TrajectoryGroup(trajectories))
print(f"Step {step+1} training")
await model.delete_checkpoints()
await model.train(train_groups, config=art.TrainConfig(learning_rate=1e-5))
(APIServer pid=1156) Step 19 rollout
(APIServer pid=1156) ....................................................................................................................................................................................
(APIServer pid=1156) s1 rewards: [0.41975308641975306, 0.012345679012345678, 0.1851851851851852, 0.654320987654321, 0.19753086419753085, 0.3950617283950617, 0.1358024691358025, 0.6172839506172839]
(APIServer pid=1156) Step 19 training
(APIServer pid=1156) No "val/reward" metric found in history
(APIServer pid=1156) Deleted checkpoint art/art-rl/models/qwen3-14b-rl/checkpoints/0017
(APIServer pid=1156) Packed 118 trajectories into 118 sequences of length 6144
train: 100%
118/118 [08:05<00:00, 4.02s/it, loss=0.0146, grad_norm=2.29, policy_loss=0.0146, entropy=0.0266]
This seems to be aligned with the code in inputs.py, which bypasses per_device_train_batch_size config.
Is there a reason to limit to one sample at a time?
- Dominant language
- Python
- Stars
- 10.8k
- Forks
- 989
- Avg merge
- 11h 38m
- Merged PRs (30d)
- 104
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenPipe/ART
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 54/100
OpenPipe/ART#961 · 3 comments ·
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
OpenPipe/ART#949 · 5 comments ·
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 42/100
Maintainers usually reply within 3 days
Similar issues
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
openvinotoolkit/openvino_notebooks#3665 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
docs
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
benchmark-gap
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day