DLR-RM/stable-baselines3

[Feature Request] independently configurable learning rates for actor and critic

Open

#338 建立於 2021年3月3日

在 GitHub 查看
 (11 留言) (1 反應) (0 負責人)Python (6,550 star) (1,407 fork)batch import
enhancementhelp wanted

描述

🚀 Feature

independently configurable learning rates for actor and critic in AC-style algorithms

Motivation

In literature the actor is often configured to learn slower, such that the critics responses are more reliable. At least it would be nice if i could allow my hyperparameter optimizer to decide which learning rates he wants to use for actor or critic.

Pitch

https://github.com/DLR-RM/stable-baselines3/blob/65100a4b040201035487363a396b84ea721eb027/stable_baselines3/ddpg/ddpg.py#L12-L26

Additional context

https://spinningup.openai.com/en/latest/algorithms/ddpg.html#documentation-pytorch-version

貢獻者指南

技術棧
pythonpytorch
領域
machine learning
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
half day
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
fresh
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
mostly clear
前置要求
understanding of reinforcement learningfamiliarity with PyTorchbasic Python
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
45
研究方向
The request is to allow separate learning rates for actor and critic in AC style algorithms. The issue points to the DDPG implementation file (ddpg.py, lines 12-26) where current learning rate handling is defined. The maintainers may need to clarify if this applies to all AC algorithms or just DDPG. Contributors should examine how the learning rate is currently passed and consider adding new parameters like `actor lr` and `critic lr` in the algorithm's constructor, then updating the optimizer setup accordingly. Reference the spinning up documentation for context on typical actor critic learning rate practices.
[Feature Request] independently configurable learning rates for actor and critic · DLR-RM/stable-baselines3#338 | Good First Issue