DLR-RM/stable-baselines3

[Feature Request] independently configurable learning rates for actor and critic

Open

#338 创建于 2021年3月3日

在 GitHub 查看
 (11 评论) (1 反应) (0 负责人)Python (6,550 star) (1,407 fork)batch import
enhancementhelp wanted

描述

🚀 Feature

independently configurable learning rates for actor and critic in AC-style algorithms

Motivation

In literature the actor is often configured to learn slower, such that the critics responses are more reliable. At least it would be nice if i could allow my hyperparameter optimizer to decide which learning rates he wants to use for actor or critic.

Pitch

https://github.com/DLR-RM/stable-baselines3/blob/65100a4b040201035487363a396b84ea721eb027/stable_baselines3/ddpg/ddpg.py#L12-L26

Additional context

https://spinningup.openai.com/en/latest/algorithms/ddpg.html#documentation-pytorch-version

贡献者指南

技术栈
pythonpytorch
领域
machine learning
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
fresh
清晰度议题是否清楚说明期望改动、验收标准和下一步。
mostly clear
前置要求
understanding of reinforcement learningfamiliarity with PyTorchbasic Python
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
45
研究方向
The request is to allow separate learning rates for actor and critic in AC style algorithms. The issue points to the DDPG implementation file (ddpg.py, lines 12-26) where current learning rate handling is defined. The maintainers may need to clarify if this applies to all AC algorithms or just DDPG. Contributors should examine how the learning rate is currently passed and consider adding new parameters like `actor lr` and `critic lr` in the algorithm's constructor, then updating the optimizer setup accordingly. Reference the spinning up documentation for context on typical actor critic learning rate practices.
[Feature Request] independently configurable learning rates for actor and critic · DLR-RM/stable-baselines3#338 | Good First Issue