thu-ml/tianshou

RNN related issues

Open

#937 opened on Sep 7, 2023

View on GitHub
 (12 comments) (0 reactions) (0 assignees)Python (1,072 forks)batch import
RNNbughelp wanted

Repository metrics

Stars
 (7,121 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

There is a bunch of reported RNN problems: #795 #513 #486 and some others. Generally, performance of agents with RNN on top never seemed to be good. This should be looked into, compared with other libraries, and fixed.

@Trinkle23897 @araffin @BFAnas If you have ideas where to start or whom to ping, feel free to write here. I guess it won't be an easy one...

Until this is solved, it might make sense to document that using RNNs is currently discouraged.

Contributor guide

Research direction
Investigate RNN performance issues in Tianshou, compare with other RL libraries (e.g., Stable Baselines3), and implement fixes based on identified root causes.
Tech stack
pythonpytorch
Domain
machine learningai
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Needs investigation
Prerequisites
PythonPyTorchDeep Reinforcement LearningRNN
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25