facebookresearch/fairseq

Add option to skip 'prepare_for_inference' to enable model training

Open

#5,155 opened on May 25, 2023

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Python (6,224 forks)batch import
enhancementhelp wantedneeds triage

Repository metrics

Stars
 (29,107 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

🚀 Feature Request

I think it should be possible to pass 'prepare_for_inference=False' into the 'from_pretrained' method from classes that inherit from 'GeneratorHubInterface'.

Motivation

It is currently not possible to turn the calling of the 'prepare_for_inference' off. It optimizes inference, but it prohibits training. I think it would be nice to be able to fine-tune these models.

Pitch

The from_pretrained methods from the affected classes already have kwargs passed to them. These kwargs could optionally contain the argument 'prepare_for_inference=False/True'. The kwargs can be added to the constructor of the affected classes and passed down to the base class constructor of GeneratorHubInterface. There, the calling of the 'prepare_for_inference' for each of the models can be conditionally turned off, if the argument 'prepare_for_inference' is False. The argument 'prepare_for_inference' should be retrieved from kwargs with 'kwargs.get("prepare_for_inference", True)' to not change the now default behaviour.

I already implemented the required changes in a fork.

Contributor guide