tracel-ai/burn

Training metrics

Open

#544 opened on Jul 25, 2023

View on GitHub
 (19 comments) (2 reactions) (0 assignees)Rust (964 forks)auto 404
featuregood first issue

Repository metrics

Stars
 (15,550 stars)
PR merge metrics
 (PR metrics pending)

Description

In burn-train, several metrics can be used during training. It would be great to have more!

  • Accuracy
  • Loss (the one in use)
  • CUDA utilization (memory&compute)
  • Top-k accuracy
  • CPU utilization
  • CPU memory usage
  • General GPU utilization
  • General GPU memory usage
  • Precision Recall
  • AUC - ROC
  • BLEU score
  • ROUGE score

Contributor guide

Research direction
Implement the missing training metrics: General GPU utilization, AUC ROC, BLEU score, and ROUGE score. Start by studying the existing metric implementations in burn train, such as accuracy and loss, then adapt the pattern for each new metric. For GPU metrics, you may need to interface with CUDA APIs. For NLP metrics like BLEU and ROUGE, refer to existing implementations in Rust or integrate with libraries.
Tech stack
rust
Domain
machine learningai
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
RustDeep Learning basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60