facebookresearch/fairseq

Print layer correlation in speech_text_joint_to_text

Open

#4386 aperta il 27 apr 2022

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Python (6224 fork)batch import
enhancementhelp wantedneeds triage

Metriche repository

Star
 (29.107 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

Note: I already implemented the code suggested by this feature request and am simultaneously making a pull request (#4387 )

🚀 Feature Request

In their paper "Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task," Tang et. al. show a graph of the correlation of decoder layers, but this code is absent from their fairseq example code. Add the necessary code to produce this data.

Motivation

This is useful for people who are trying to reproduce the paper and show similar results to the authors' data.

Pitch

For every epoch, fairseq should print out the correlation data for each layer when training the speech_text_joint_to_text task. Ideally, both validation and training phases will be included.

Alternatives

-Ask the authors for their code/method -Dump all (or a sample) the parameters periodically and use a post-processing tool to collect the data

Additional context

Ideally, this should be computed on the GPU for quick turnaround.

Guida contributor