[Feature Request] Resume study and avoid OOM when optimizing with Optuna plugin
#1.679 aberto em 17 de jun. de 2021
Métricas do repositório
- Stars
- (7.539 stars)
- Métricas de merge de PR
- (Mesclagem média 37d 6h) (25 fundiu PRs em 30d)
Description
🚀 Feature Request
I would like to be able:
- to persist my study in order to resume my hyperparameter search from where I left it
- set the
gc_after_trialparameter of Optuna'sstudy.optimize()
Motivation
Is your feature request related to a problem? Please describe.
I'm always frustrated when my code crashes after 60 trials (out of 100). I suspect an OOM error. Being able to prevent the script from crashing in the first place with gc.collect() would be great. However, at least being able to resume my search from where it stopped would be a game changer.
Pitch
Describe the solution you'd like
I would like to set gc_after_trial to True and a path to store my study parameters after each trial in my Optuna sweeper hydra config.
Describe alternatives you've considered I read Optuna's documentation but I'm not sure how to make their examples work with Hydra:
- How do I avoid running out of memory (OOM) when optimizing studies?
- How can I save and resume studies?
Are you willing to open a pull request? (See CONTRIBUTING) I'm not comfortable enough with Optuna's and Hydra's library to prepare a pull request.