Label-wise metrics (Accuracy etc.) for multi-label problems
#513 ouverte le 2 mai 2019
Métriques du dépôt
- Stars
- (4 313 stars)
- Métriques de merge PR
- (Merge moyen 18j 23h) (19 PRs mergées en 30 j)
Description
Hi,
I've made a multi-label classifier using BCEWithLogitsLoss. In summary a data sample can be one of 3 binary classes, which aren't mutually eclusive, so y_pred and y can look something like [0, 1, 1].
My metrics include Accuracy(output_transform=thresholded_output_transform, is_multilabel=True) and Precision(output_transform=thresholded_output_transform, is_multilabel=True, average=True)}.
However, I'm interesting in having label-specific metrics (i.e. having 3 accuracies etc.). This is important because it allows me to see what labels are compromising my overall accuracy the most (a 70% accuracy be a 30% error in a single label, or a more modest error scattered across 3 labels).
There is no option to disable averaging for Accuracy() as with the others, and setting average=False for Precision() does not do what I expected (it yields a binary result per datum, not per label, so I end up with a tensor of size 500, not 3, if my dataset n=500).
Is there a way to get label-wise metrics in mutlilabel problems? Or a plan to introduce it?
P.S. I'd love to get an invite to the slack workspace if possible? How do I go about doing that?