描述
This is definitely a nice to have -- It'd be nice if as a part of snapshots, we could have the option to record the policy being rolled out deterministically for a few rollouts, so that we can watch its behavior in our various environments. I haven't had much success using the out of the box tool that open ai gym provides for this, called the gym Monitor wrapper. It requires that all environments internally return a Done signal after max_path_length steps. Perhaps we can get around this by wrapping environments with the open ai gymtimelimit wrapper.
Alternatively someone could look at the stable baselines vec video recorder wrapper. It seems that they have some type of video recorder, however it doesn't work out of the box with garage since stable baselines has pinned versions of TF and other libraries necessary to garage.