facebookresearch/fairseq

Print layer correlation in speech_text_joint_to_text

Open

#4,386 创建于 2022年4月27日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Python (6,224 fork)batch import
enhancementhelp wantedneeds triage

仓库指标

Star
 (29,107 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

Note: I already implemented the code suggested by this feature request and am simultaneously making a pull request (#4387 )

🚀 Feature Request

In their paper "Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task," Tang et. al. show a graph of the correlation of decoder layers, but this code is absent from their fairseq example code. Add the necessary code to produce this data.

Motivation

This is useful for people who are trying to reproduce the paper and show similar results to the authors' data.

Pitch

For every epoch, fairseq should print out the correlation data for each layer when training the speech_text_joint_to_text task. Ideally, both validation and training phases will be included.

Alternatives

-Ask the authors for their code/method -Dump all (or a sample) the parameters periodically and use a post-processing tool to collect the data

Additional context

Ideally, this should be computed on the GPU for quick turnaround.

贡献者指南