[Lane 1 · numerics] Precision/Recall/F1 — averaging-mode and zero-division conventions

Fermée
  • Quelque chose ne va pas ?
#1,224 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
Une demi-journée
Accessibilité débutants
68/100
Type d'issue
Documentation
Clarté
Clairement spécifiée
Activité
Calme
Stack technique
python, pytorch, scikit-learn
Domaine
ai, machine-learning

Piste de recherche

This is research only: no Kotlin or SKaiNET code. Read sklearn.metrics.precision_recall_fscore_support (docs and source) and torchmetrics.classification Precision, Recall, and F1Score. Build the edge-case table (empty batch, unseen class, one-sided class, all wrong, argmax ties) for macro and micro, then write three short recommendations (zero-division, macro class set, binary/threshold). Confirm the given fixture with a real sklearn call and post table plus recommendations as a comment on issue #1222; done when a maintainer replies adopted or amended.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

good first issue research size:s skill:numerics sub-issue training

Sub-issue of #1222 (Precision, Recall and F1Score metrics).

Lane: 1 · Numerics / Research
Skill needed: comfortable reading scikit-learn / torchmetrics source and docs. No Kotlin required, no SKaiNET knowledge required.
Size: s (~2–4 h)
Blocked by: nothing — this is the first task in the feature and unblocks #1226.

What to do

  1. Read sklearn.metrics.precision_recall_fscore_support (docs + source) and
    torchmetrics.classification.{Precision, Recall, F1Score} (docs + source).
  2. Produce a short table — one row per edge case, one column per library — for:
    • empty batch (no samples at all);
    • a class that appears in neither predictions nor targets (TP+FP+FN = 0);
    • a class that appears only in targets (never predicted) and vice versa;
    • all predictions wrong;
    • ties in the argmax (two logits exactly equal).
      For each: what does the library return for precision / recall / F1 under macro and
      micro, and do the two libraries agree?
  3. Answer the three decisions the parent issue leaves open, each as a one-paragraph
    recommendation with a link a reviewer can click:
    • Zero-division: 0.0 (sklearn default) vs. NaN vs. a parameter. The parent
      recommends 0.0 to match Accuracy.compute() on an empty accumulator.
    • Class set for macro averaging: union of classes seen in targets ∪ predictions
      (sklearn) vs. the full class dimension predictions.shape[dim] (which counts
      never-seen classes as F1 = 0). These differ on small batches.
    • Binary mode: does binary mean "report the positive class (index 1) only", and
      should F1Score take a threshold: Float? like Accuracy does, or is binary purely
      averaging = BINARY?
  4. Post the table and the three recommendations as a comment on #1222 under
    its Research section. That comment is the deliverable — not code.

Acceptance

  • Edge-case table posted on #1222 with links to the exact sklearn / torchmetrics
    doc sections or source lines
  • Three explicit recommendations (zero-division, macro class set, binary/threshold)
  • A maintainer has replied "adopted" (or amended) so #1226 can start from a fixed contract

Notes

The hand-computed reference fixture the Kotlin sub-issues will test against is:
predicted classes [2, 0, 1, 2], targets [2, 1, 1, 0] (3 classes). Expected under
sklearn: per-class precision [0, 1, 0.5], recall [0, 0.5, 1], F1 [0, 0.667, 0.667];
macro P = R = 0.5, macro F1 = 0.444; micro P = R = F1 = 0.5 (= accuracy). Please confirm
those numbers by actually running precision_recall_fscore_support once and paste the
output — that turns a hand calculation into a citable reference.

Langage dominant
Kotlin
Étoiles
52
Forks
15
Merge moyen
23 h 32 min
PR mergées (30 j)
81

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de SKaiNET-developers/SKaiNET

Toutes les issues de SKaiNET-developers/SKaiNET

Issues similaires

Plus d'issues Kotlin

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.