Lightning-AI/pytorch-lightning

Provide a non-expert mode for the DeviceStatsMonitor

Open

#18,652 建立於 2023年9月27日

在 GitHub 查看
 (0 留言) (15 反應) (0 負責人)Python (3,233 fork)batch import
callback: device statsfeaturehelp wanted

倉庫指標

Star
 (26,687 star)
PR 合併指標
 (平均合併 9天 15小時) (30 天內合併 3 個 PR)

描述

Description & Motivation

The DeviceStatsMonitor currently dumps the entire torch.cuda.memory_stats into the logger. Beginner users might be overwhelmed by the amount of data logged and are confused which metrics matter to maximize their utilization.

Pitch

Make the callback more user-friendly by logging only core metrics like GPU utilization and total memory usage. Provide an expert mode that when enabled logs what we see today.

Alternatives

No response

Additional context

This was proposed by user magical_arjay on our Discord.

cc @borda @awaelchli

貢獻者指南