NMI/AMI return 0.0 instead of 1.0 for two single-cluster labelings

Open Beginner friendly

#3,469 opened on Aug 20, 2026 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 2/5
Estimated time: 1-3 hours
Newbie friendliness: 75/100

Issue type: Bug
Clarity: Clearly specified
Activity status: Active
Tech stack: python, pytorch
Domain: machine-learning

Research direction

Start with the normalized_mutual_info_score and adjusted_mutual_info_score implementations in torchmetrics.functional.clustering, then run the provided reproduction against the current behavior. Done means identical single-cluster labelings return 1.0 for both metrics while the asymmetric single-cluster case remains 0.0, matching the documented scikit-learn reference.

Written by the indexing model from the issue text.

Description

Bug

normalized_mutual_info_score and adjusted_mutual_info_score return 0.0 for two identical single-cluster labelings, where scikit-learn (the documented reference for these metrics) returns 1.0, a perfect match.

Repro

import torch, numpy as np
import sklearn.metrics as skm
import torchmetrics.functional.clustering as tmc

a = np.array([0, 0, 0, 0]); ta = torch.tensor(a)
print(float(tmc.normalized_mutual_info_score(ta, ta)), skm.normalized_mutual_info_score(a, a))  # 0.0 vs 1.0
print(float(tmc.adjusted_mutual_info_score(ta, ta)), skm.adjusted_mutual_info_score(a, a))        # 0.0 vs 1.0

Why

Both labelings have zero entropy, so they trivially agree. scikit-learn treats this limit case as a perfect match and returns 1.0. torchmetrics falls through to the mutual_info == 0 short-circuit and returns 0.0. The asymmetric case (one single cluster, the other not) already returns 0.0 in both libraries and is correct.

Environment: torchmetrics 1.9.0 / current master.

I have a fix ready and will open a PR referencing this issue.

Dominant language: Python
Stars: 2.5k
Forks: 526
Avg merge: 8d 4h
Merged PRs (30d): 7

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.