GluonTSFramework Hyper Parameter Optimization Support · awslabs/gluonts#617

(0 comments) (4 reactions) (0 assignees)Python (3,888 stars) (753 forks)batch import

enhancementhelp wanted

Description

Implement Hyper Parameter Optimisation (HPO) Support in GluonTSFramework. There are already multiple comments in the code of how to go about it:

# HPO implementation sketch:
#    > Example HPO of model: MODEL_HPM:Trainer:batch_size:64
#    > Now construct nested dict from MODEL_HPM hyperparameters
#    > Load the serialized model as a dict
#    > Update the model dict with the nested dict from the MODEL_HPMs
#      with dict.update(...)
#    > Write this new dict back to a s3 as a .json file like before

This is important to support:

The HyperparameterTuner: https://sagemaker.readthedocs.io/en/stable/tuner.html
and SageMaker Experiments: https://aws.amazon.com/blogs/aws/amazon-sagemaker-experiments-organize-track-and-compare-your-machine-learning-trainings/

References

Where the https://github.com/awslabs/gluon-ts/blob/master/src/gluonts/nursery/sagemaker_sdk/estimator.py

Contributor guide

Tech stack: pythonaws
Domain: machine learningbackendcloud
Issue type: feature
Difficulty: 4
Estimated time: 3-5 days
Activity status: stale
Clarity: mostly clear
Prerequisites: PythonAWS SageMakertime series modelingGluonTS basics
Newbie friendliness: 20
Research direction: Examine the implementation sketch in the issue and the referenced file `src/gluonts/nursery/sagemaker sdk/estimator.py` to understand the current structure. Then, investigate how GluonTS currently handles hyperparameters and look at existing SageMaker HPO patterns in the AWS SDK documentation. The goal is to implement a mechanism to parse hyperparameter strings like 'MODEL HPM:Trainer:batch size:64' into nested dictionaries and update the serialized model config accordingly. Consider checking linked issues or PRs for prior attempts.