pytorch/ignite

Top-K precision/recall multilabel metrics for ranking task

Open

#467 opened on Mar 31, 2019

View on GitHub
 (12 comments) (0 reactions) (0 assignees)Python (602 forks)batch import
enhancementhelp wantedmodule: metrics

Repository metrics

Stars
 (4,313 stars)
PR merge metrics
 (Avg merge 18d 23h) (19 merged PRs in 30d)

Description

Following the discussion from https://github.com/pytorch/ignite/issues/466#issuecomment-478339986 it would be nice to have such metric in Ignite.

In the context of a multilabel task, compute a top-k precision/recall per label (treating all labels independently).

Contributor guide

Research direction
Examine the discussion in linked issue #466 and the existing metric implementations in Ignite to understand the pattern for adding a new metric. Focus on the expected behavior for top k precision/recall per label in a multilabel ranking context.
Tech stack
pythonpytorch
Domain
machine learningai
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
PythonPyTorchIgnite
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60