Have `train!` record, for inclusion in output, a history of the public state of every control

Open
#64 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@ablaom is already working on this.

Since Oct 7, 2026.

Assessment

This issue has not been assessed yet.

Description

I propose some enhancements to address two main shortcomings of IterationControl.jl, mostly based on working on tutorials for MLJIteration.jl (which wraps MLJ models in the controls provided by IterationControl.jl):

  • While possible in practice, it is rather complicated to do things like: (i) Get a history of losses (or training losses) for training over multiple iterations (see #47); and (i) Set up a callback to record, say, evolution of some learned parameters. In both cases, you have to define and initialise containers to store the relevant trace, and also to arrange the relevant callback control to write to that container as training proceeds. When you repeat the experiment, you may need to flush the trace containers. This is inconvenient.

  • The internal state of a control is currently considered private API and the only interface point is for accessing what, from the final state, is returned in the "report" output by "train!". This seems unnecessarily opaque.

  • The "report" returned by "train!" is quite hard to grok, without delving into docs or scratching your head.

I propose:

  • Introduce a new method public_state(control, state) that exposes everything about state that might be useful to know at every control cycle, in a human readable form (named tuple, say)

  • By default, have train! record a complete history of the public state of each control, exposed in the report, with a compact=true option to just record final state (for the atypical case fallbacks are exposing large objects as part of state)

  • Make improvements to the structure and display of the report to make access more intuitive.

Among other things, these changes would likely address #62 and #47.

These changes would be necessarily breaking, although the logic of controls themselves would not change much, if at all.

Some steps required are:

  • Introduce public_state the more primitive level of EarlyStopping.jl (a dep of IterationControl.jl).
Dominant language
Julia
Stars
25
Forks
1
Avg merge
2d 8h
Merged PRs (30d)
6

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaAI/IterationControl.jl

All issues in JuliaAI/IterationControl.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.