Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Feature]: Precision, Recall and F1Score metrics

Aperta
  • Qualcosa non va?
#1,222 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
kotlin
Ambito
machine-learning

Direzione di ricerca

This is an umbrella issue: implementation is split across #1224–#1235, so start by checking those issues and their dependencies. For the Kotlin core, read skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/metrics/Accuracy.kt and resolve the averaging and constructor questions in #1224 before #1226 starts. The parent is done when the listed acceptance criteria are met, including the three metrics, unit tests, API dump, documentation, and review.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

darc enhancement tracking training

🧠 D: DOCUMENT — Problem & Opportunity

sk.ainet.lang.nn.metrics has exactly one metric: Accuracy. Precision, recall and F1 —
the next three metrics any classifier evaluation needs, and the ones that actually say
something on imbalanced data — are missing. Every framework SKaiNET is compared against
or ported from (KotlinDL, PyTorch/torchmetrics, scikit-learn, Keras) ships all four as the
baseline set; today a SKaiNET training loop has to leave the library to compute them.

Summary: Add Precision, Recall and F1Score to nn/metrics/, built on one
shared per-class confusion-matrix accumulator, following the existing Metric interface
(update / compute / reset) and Accuracy's dtype / dim / threshold handling.

Design decision (Assess, made here so Code doesn't have to): F1 depends on precision
and recall, which are also missing. Three classes each re-deriving TP/FP/FN would
triplicate the non-trivial iteration logic already in
skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/metrics/Accuracy.kt
(dtype dispatch, hard vs. soft targets, argmax over dim, binary threshold). So: extract
Accuracy's private helpers once, build one internal ConfusionMatrixAccumulator, and
let the three metrics be three compute() formulas over it. Ship all three from one DARC
cycle — same data, three views.

DARC or SKEEP? DARC only. These are additive classes behind the existing Metric
interface: no public-API-shape, DSL, storage, or compiler footprint. The one part of this
work that does trip a SKEEP trigger — teaching the ground-truth harness to validate
stateful metrics, not just stateless tensor ops — is split out as its own proposal
(#1223) rather than being settled inside a sub-issue. See the docs page
Contributing → Worked example: F1Score via DARC for the full reasoning.

🔍 A: ASSESS — Feasibility & Impact

✔️ Feasibility

Straightforward for binary classification (one confusion matrix). Multi-class needs an
explicit averaging-strategy decision (macro / micro / per-class) — that is where the design
risk lives, not the arithmetic.

✔️ Expected Impact

Unblocks any training/eval loop that needs more than raw accuracy — most imbalanced
classification workloads — and closes a gap against every framework in the comparison set.

✔️ Risks / Constraints
  • Zero-division: F1 is undefined when TP+FP+FN = 0 for a class. sklearn defaults to
    0.0 with a warning; torchmetrics has a zero_division parameter. Pick one, document
    it, never leave it as an unspecified NaN. Recommendation: return 0.0, matching
    Accuracy.compute()'s empty-accumulator behaviour.
  • Averaging semantics: macro (unweighted mean over classes) vs. micro (pooled TP/FP/FN)
    vs. per-class give different numbers for the same predictions. A cross-check that falls
    out of the definitions: for single-label multi-class via argmax, micro precision ==
    micro recall == accuracy     — worth a test.
  • Which classes does "macro" average over? sklearn averages over the union of labels
    present in y_true ∪ y_pred; averaging over the full class dimension (preds.shape[dim])
    gives a different number when a class never appears in a batch. Needs a decision
    (#1224).
  • Ground-truth validation has no home yet: OperationExecutor in
    skainet-test-groundtruth maps a name to a stateless TensorOps call and returns one
    tensor; a Metric is stateful and returns a scalar. This is a process risk, resolved
    by shipping v1 with unit tests only and tracking the harness extension as SKEEP-005
    (#1223).
✔️ Dependencies

Nothing new. Built entirely from tensor element access already used by Accuracy.

📚 R: RESEARCH — What Must Be Understood First?

Research Tasks
  • Reference implementations: sklearn.metrics.precision_recall_fscore_support,
    torchmetrics.classification.{Precision,Recall,F1Score}
  • Formula: F1 = 2·P·R / (P + R), per class; macro = mean over classes; micro = pooled
  • Averaging modes for v1 — recommend binary + macro + micro, per-class (no
    reduction) as a follow-up — #1224
  • Zero-division convention — recommend 0.0 — #1224
  • Class set for macro averaging (seen classes vs. full class dim) — #1224
Open Questions

Does F1Score take a threshold: Float? mirroring Accuracy, or is binary handled
entirely by averaging = Averaging.BINARY? Affects the public constructor shape; resolve
in #1224 before #1226 starts.

🛠️ C: CODE — Lane breakdown

Lanes per Contributing → Issue taxonomy. Lane 0 (SKEEP) is skipped for the metrics
themselves and lives in #1223; Lane 4 (ground-truth wiring) is deliberately not a
sub-issue here — it becomes actionable only once SKEEP-005 is accepted.

Sub-issues

Lane Issue Skill Size Entry point Blocked by
1 · Numerics research #1224 skill:numerics s good first issue — no Kotlin —
2 · Kotlin core: extract Accuracy helpers #1225 skill:kotlin-core s good first issue —
2 · Kotlin core: Averaging + ConfusionMatrixAccumulator #1226 skill:kotlin-core s #1224, #1225
2 · Kotlin core: Precision #1227 skill:kotlin-core s good first issue #1226
2 · Kotlin core: Recall #1228 skill:kotlin-core s good first issue #1226
2 · Kotlin core: F1Score #1229 skill:kotlin-core s good first issue #1227, #1228
3 · Platform: Android #1230 skill:android s good first issue #1229
3 · Platform: iOS #1231 skill:ios xs good first issue #1229
3 · Platform: JS/Wasm #1232 skill:js xs good first issue #1229
3 · Platform: Kotlin/Native #1233 skill:native xs good first issue #1229
4 · Ground-truth / CI — — — not a sub-issue — see SKEEP-005 #1223 SKEEP-005 accepted
5 · Docs partials #1234 skill:docs s good first issue (#1229 for the examples tag)
6 · DARC review #1235 skill:review s #1229, #1234
0 · Design (SKEEP) #1223 skill:design m separate tracking issue, not a sub-issue —

To claim a lane: comment on the sub-issue; a maintainer assigns it. Report results back here when it closes.

Acceptance Criteria
  • Precision, Recall, F1Score + factory functions in sk.ainet.lang.nn.metrics, API dump updated
  • Unit tests: hand-computed binary, macro and micro multi-class cases, zero-division edge case, micro == accuracy cross-check
  • Doc partials under docs/modules/ROOT/partials/ops/metrics/, included from the metrics how-to
  • Reviewed by someone other than the implementer; @DarcValidated added post-review
  • Ground-truth coverage explicitly stated as "unit tests only, pending SKEEP-005" in the PRs — a reviewed trade-off, not an oversight
Lingua principale
Kotlin
Stelle
52
Fork
15
Merge medio
1g 15h
PR unite (30g)
36

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di SKaiNET-developers/SKaiNET

Tutte le issue di SKaiNET-developers/SKaiNET

Issue simili

Altre issue su Kotlin

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.