pytorch/ignite

Improve distributed tests for Confusion Matrix (CM and MLCM)

Open

Aperta il 20 feb 2021

Vedi su GitHub
 (6 commenti) (0 reazioni) (0 assegnatari)Python (4313 star) (602 fork)batch import
HacktoberfestPyDataGlobalenhancementhelp wantedmodule: metrics

Descrizione

Currently, the testing data is generated without taking into account distributed context: https://github.com/pytorch/ignite/blob/e17acc740d38909ddbbed6648423750abe593e58/tests/ignite/metrics/test_confusion_matrix.py#L559-L565

same for MultiLabelConfusionMatrix tests.

We would like to rewrite the tests in a similar way as here : https://github.com/pytorch/ignite/blob/e17acc740d38909ddbbed6648423750abe593e58/tests/ignite/metrics/test_recall.py#L778-L785

See https://github.com/pytorch/ignite/issues#issuecomment-782750122

cc @touqir14

Guida contributor

Tech stack
pythonpytorch
Dominio
testingmachine learning
Tipo issue
test
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
1-3 hours
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
clear
Prerequisiti
PyTorch basicsIgnite test structuredistributed testing knowledgeconfusion matrix metrics
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
60
Direzione di ricerca
The issue requires rewriting the test data generation for ConfusionMatrix and MultiLabelConfusionMatrix to properly simulate distributed context. The existing tests in tests/ignite/metrics/test confusion matrix.py (lines 559-565) need to be replaced with a pattern similar to tests/ignite/metrics/test recall.py (lines 778-785). The contributor should study the distributed test setup in the linked example and ensure the metrics compute correctly across multiple processes.