CarperAI/trlx

Add tensor padding options to improve hardware utilization

Open

#284 建立於 2023年2月6日

在 GitHub 查看
 (8 留言) (2 反應) (0 負責人)Python (450 fork)batch import
contributions welcomefeature requestgood first issue

倉庫指標

Star
 (4,184 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

🚀 The feature, motivation, and pitch

Recent discussion on Twitter highlighted the importance of tensor padding to improve hardware util. NeMo seems to already support GPU-friendly vocab size padding from discussions with @cat-state but we should also consider adding an optional pad_to_multiple_of arg to our tokenizer calls with the accelerate/transformers backend to satisfy Tensor Core requirements.

image

The implementer should also provide system plots to display any improvements/findings.

Alternatives

No response

Additional context

No response

貢獻者指南