CarperAI/trlx

Add tensor padding options to improve hardware utilization

Open

#284 创建于 2023年2月6日

在 GitHub 查看
 (8 评论) (2 反应) (0 负责人)Python (450 fork)batch import
contributions welcomefeature requestgood first issue

仓库指标

Star
 (4,184 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

🚀 The feature, motivation, and pitch

Recent discussion on Twitter highlighted the importance of tensor padding to improve hardware util. NeMo seems to already support GPU-friendly vocab size padding from discussions with @cat-state but we should also consider adding an optional pad_to_multiple_of arg to our tokenizer calls with the accelerate/transformers backend to satisfy Tensor Core requirements.

image

The implementer should also provide system plots to display any improvements/findings.

Alternatives

No response

Additional context

No response

贡献者指南