facebookresearch/metaseq

Add Model FLOPS Utilization and Hardware FLOPS utilization % to the logs.

Open

#246 opened on Jul 22, 2022

View on GitHub
 (2 comments) (6 reactions) (0 assignees)Python (6,195 stars) (701 forks)batch import
better-engenhancementgood first issuetelemetry

Description

Calculation on page 65 of palm paper: https://arxiv.org/pdf/2204.02311.pdf

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic PythonPyTorch basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Review the existing logging infrastructure in metaseq, likely in the training loop. Compute Model FLOPS Utilization and Hardware FLOPS utilization % as described in the PaLM paper (page 65). Implement the calculation and add the metrics to the logs. Check if any related PRs or issues exist for reference.