Print layer correlation in speech_text_joint_to_text
#4,386 opened on Apr 27, 2022
Repository metrics
- Stars
- (29,107 stars)
- PR merge metrics
- (No merged PRs in 30d)
Description
Note: I already implemented the code suggested by this feature request and am simultaneously making a pull request (#4387 )
🚀 Feature Request
In their paper "Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task," Tang et. al. show a graph of the correlation of decoder layers, but this code is absent from their fairseq example code. Add the necessary code to produce this data.
Motivation
This is useful for people who are trying to reproduce the paper and show similar results to the authors' data.
Pitch
For every epoch, fairseq should print out the correlation data for each layer when training the speech_text_joint_to_text task. Ideally, both validation and training phases will be included.
Alternatives
-Ask the authors for their code/method -Dump all (or a sample) the parameters periodically and use a post-processing tool to collect the data
Additional context
Ideally, this should be computed on the GPU for quick turnaround.