[Feature Request] independently configurable learning rates for actor and critic · DLR-RM/stable-baselines3#338

(11 comments) (1 reaction) (0 assignees)Python (6,550 stars) (1,407 forks)batch import

enhancementhelp wanted

説明

🚀 Feature

independently configurable learning rates for actor and critic in AC-style algorithms

Motivation

In literature the actor is often configured to learn slower, such that the critics responses are more reliable. At least it would be nice if i could allow my hyperparameter optimizer to decide which learning rates he wants to use for actor or critic.

Pitch

https://github.com/DLR-RM/stable-baselines3/blob/65100a4b040201035487363a396b84ea721eb027/stable_baselines3/ddpg/ddpg.py#L12-L26

Additional context

https://spinningup.openai.com/en/latest/algorithms/ddpg.html#documentation-pytorch-version

コントリビューターガイド

技術スタック: pythonpytorch
領域: machine learning
Issue 種別: feature
難度: 3
推定時間: half day
活動状況: fresh
明確さ: mostly clear
前提条件: understanding of reinforcement learningfamiliarity with PyTorchbasic Python
初心者向け度: 45
調査方針: The request is to allow separate learning rates for actor and critic in AC style algorithms. The issue points to the DDPG implementation file (ddpg.py, lines 12-26) where current learning rate handling is defined. The maintainers may need to clarify if this applies to all AC algorithms or just DDPG. Contributors should examine how the learning rate is currently passed and consider adding new parameters like `actor lr` and `critic lr` in the algorithm's constructor, then updating the optimizer setup accordingly. Reference the spinning up documentation for context on typical actor critic learning rate practices.