Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

EmbeddingCollapseMetric: per-class rank scores never reach `aggregate`, so asymmetric collapse is invisible in the summary score

Aperta
  • Qualcosa non va?
#9,116 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@DucAnhValentinoNguyen ci sta già lavorando.

Dal 12/9/2026.

  • #9117 di @DucAnhValentinoNguyen — aperta

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, pytorch
Ambito
machine-learning

Direzione di ricerca

Inizia in monai/metrics/embedding_collapse.py, in compute_embedding_collapse e _per_class_rank, quindi esegui la riproduzione ed esamina test_healthy_embeddings_score_low. Il lavoro è completo quando il rango per classe contribuisce all’aggregato secondo la semantica di riduzione scelta, con test che coprano il comportamento di riepilogo interessato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug

In compute_embedding_collapse (monai/metrics/embedding_collapse.py), the aggregate score is built from a fixed key set:

primary = {"centroid_similarity", "effective_rank_score", "domain_shift", "separation"}
available = [v.to(device=emb.device) for k, v in scores.items() if k in primary and v is not None]

_per_class_rank emits its scores under per_class_rank_<cls>, so no per-class score is ever a member of primary, and none is ever included in available. The indicator is computed, returned in the dict, and then dropped before every reduction.

That is the one indicator whose documented purpose is catching collapse the global indicators structurally cannot see:

Detects asymmetric collapse: one class may use 400 dimensions while another collapses to 6, which global SVD would average away.

So under reduction="max" — documented as "the worst-case score (recommended for safety-critical use)" — the worst case is precisely the number that gets discarded. A caller who follows the docstring and monitors aggregate gets no signal from it at all.

To Reproduce

A healthy majority class, and a rare class whose encoder output has collapsed to a single point:

import torch
from monai.metrics.embedding_collapse import compute_embedding_collapse

torch.manual_seed(17)

n_major, n_minor, d = 200, 8, 64

major = torch.randn(n_major, d)
major[:, 0] -= 4.0          # centroid ~ -e1

minor = torch.zeros(n_minor, d)
minor[:, 0] = 8.0           # 8 identical rows, centroid = +e1

emb = torch.cat([major, minor])
lbl = torch.tensor([0] * n_major + [1] * n_minor, dtype=torch.long)

scores = compute_embedding_collapse(emb, lbl, reduction="max")
for k, v in scores.items():
    print(f"{k:24s} {float(v) if v is not None else None}")

Measured output:

centroid_similarity      0.0057
per_class_rank_0         0.0485
per_class_rank_1         1.0000
separation               0.2658
effective_rank_score     0.0602
domain_shift             None
aggregate                0.2658

Class 1 has exactly zero within-class variance, so per_class_rank_1 is 1.0 — the module's own maximum, via the sv_sum == 0 branch in _effective_rank_score. Yet aggregate remains 0.2658.

The global indicators are not misbehaving here. 200 of 208 samples are full-rank, so the global spectrum is barely perturbed (effective_rank_score == 0.0602); the centroids are anti-parallel, so centroid_similarity sits at the bottom of its range (0.0057). Every global view of this embedding space really is healthy. Only the per-class view sees that one class is dead — and that view never reaches the summary.

This is independent of #8996: every effective-rank call in this example has N > D (or exits via the zero-variance branch), so the 1/N floor does not apply.

Expected behavior

Per-class rank should participate in the reduction. Two ways to do it, and I would like a steer on which you guys prefer:

  1. Add each per_class_rank_<cls> to available individually. Correct under max. Under mean it re-weights the average by class count — C per-class terms against 4 global ones — so the meaning of the mean drifts as C grows.
  2. Reduce per-class scores to one representative value first (max over classes, i.e. "the worst class"), then add that single value to available. Identical to option 1 under max; keeps one vote per indicator under mean, regardless of C.

I have implemented option 2 in a PR, since it leaves reduction="mean" interpretable, but option 1 is a small change if you would rather every class count separately.

One note on the existing tests

test_healthy_embeddings_score_low builds each class from identical rows (+e1 x 10, -e1 x 10), so under either option both classes score per_class_rank == 1.0 and that fixture's aggregate becomes 1.0. The test only asserts on centroid_similarity, so it still passes — but the fixture is a genuinely zero-within-class-variance embedding space, so the new score is correct and the name is what is misleading. Happy to either give it within-class variance or rename it in the same PR, whichever you prefer.

Environment

dev branch (MONAI 1.6.0+).

Lingua principale
Python
Stelle
8.7k
Fork
1.6k
Merge medio
5g 3h
PR unite (30g)
14

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Project-MONAI/MONAI

Tutte le issue di Project-MONAI/MONAI

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.