thu-ml/tianshou

RNN related issues

Open

#937 opened on Sep 7, 2023

View on GitHub
 (12 comments) (0 reactions) (0 assignees)Python (7,121 stars) (1,072 forks)batch import
RNNbughelp wanted

Description

There is a bunch of reported RNN problems: #795 #513 #486 and some others. Generally, performance of agents with RNN on top never seemed to be good. This should be looked into, compared with other libraries, and fixed.

@Trinkle23897 @araffin @BFAnas If you have ideas where to start or whom to ping, feel free to write here. I guess it won't be an easy one...

Until this is solved, it might make sense to document that using RNNs is currently discouraged.

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningai
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
RNN in RLTianshou codebasePythonPyTorch
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Investigate the performance issues with RNN based agents in Tianshou by examining the relevant code in the repository, likely in the policy or network modules. Compare with implementations in other RL libraries such as Stable Baselines3 to identify bottlenecks. The issue references #795, #513, and #486, which may contain specific reports and should be reviewed for context.