facebookresearch/fairseq

reduce padding overhead when using buckets

開放

#5,011 建立於 2023年3月6日

 (1 則留言) (1 個反應) (0 位負責人)Python (6,224 個分叉)batch import
enhancementhelp wantedneeds triage

倉庫指標

星標
 (29,107 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

🚀 Feature Request

The current get_buckets function ensures that you have the same number of samples in each bucket. This can result in many unnecessary padding. The feature would enable the user to set an optimal bucket size to reduce padding.

Motivation

padding = unrequited work -> bad performance. While finding the optimal bucket can take 1-5 minutes it can reduce training time by 10-50%.

Pitch

Add support for using Integer programming to set the optimal bucketing size. The implementation I had in mind depends on pulp. I can share the code for optimal bucketing.

Alternatives

k-means algorithm can give a suboptimal but faster approximation

Additional context

貢獻者指南