How to save val_data predictions during each evaluation of `trainer.fit`?

Open
#401 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
machine-learning

Research direction

Start by tracing the trainer.fit evaluation flow and the val_data handling mentioned in the issue, then identify where the final State is produced. Check whether the existing API exposes predictions for each evaluation step and document the supported way to access and save them, including what completion looks like for repeated evaluations.

Written by the indexing model from the issue text.

Description

enhancement question v0.x verl

During the trainer.fit process, how can I access and save the prediction results (for example, the final State returned) of the val_data for each evaluation step?

Dominant language
Python
Stars
18.4k
Forks
1.6k
Avg merge
2d 8h
Merged PRs (30d)
27

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from microsoft/agent-lightning

All issues in microsoft/agent-lightning

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.