Open
Integrate LucidRain's RotaryEmbeddings
enhancementgood first issue
2 comments0 reactions0 assignees
Issues
Open
Remove apex dependency after PyTorch 2.0?
better-enggood first issue
3 comments0 reactions1 assignee
Open
Remove Megatron dependency - move entirely to Fairscale
better-enggood first issue
1 comment0 reactions1 assignee
Open
Convert string matches to enum comparison if possible
better-engconfigenhancementgood first issue
4 comments0 reactions0 assignees
Open
Unify tokenizers
better-engenhancementgood first issue
1 comment1 reaction1 assignee
Open
Add Model FLOPS Utilization and Hardware FLOPS utilization % to the logs.
better-engenhancementgood first issuetelemetry
2 comments6 reactions0 assignees
Open
Track all of our RNG offsets to avoid collisions
enhancementgood first issue
0 comments0 reactions0 assignees
Open
Add tests for MultiplePadDataset class
better-engenhancementgood first issue
2 comments0 reactions0 assignees