facebookresearch/fairseq

Add option to skip 'prepare_for_inference' to enable model training

Open

#5155 aperta il 25 mag 2023

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Python (6224 fork)batch import
enhancementhelp wantedneeds triage

Metriche repository

Star
 (29.107 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

🚀 Feature Request

I think it should be possible to pass 'prepare_for_inference=False' into the 'from_pretrained' method from classes that inherit from 'GeneratorHubInterface'.

Motivation

It is currently not possible to turn the calling of the 'prepare_for_inference' off. It optimizes inference, but it prohibits training. I think it would be nice to be able to fine-tune these models.

Pitch

The from_pretrained methods from the affected classes already have kwargs passed to them. These kwargs could optionally contain the argument 'prepare_for_inference=False/True'. The kwargs can be added to the constructor of the affected classes and passed down to the base class constructor of GeneratorHubInterface. There, the calling of the 'prepare_for_inference' for each of the models can be conditionally turned off, if the argument 'prepare_for_inference' is False. The argument 'prepare_for_inference' should be retrieved from kwargs with 'kwargs.get("prepare_for_inference", True)' to not change the now default behaviour.

I already implemented the required changes in a fork.

Guida contributor