FedML-AI/FedML

A bug encountered when using fed_cifar100 in centralized settings.

Open

#805 aperta il 6 mar 2023

Vedi su GitHub
 (2 commenti) (0 reazioni) (0 assegnatari)Python (743 fork)batch import
help wanted

Metriche repository

Star
 (3930 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

When using fed_cifar100 in centralized settings, I encountered a bug. In the file fedml/data/data_loader.py, line 559 constructs the test_data_local_dict in the following way:

test_data_local_dict = {
            0: [batch for cid in sorted(test_data_local_dict.keys()) for batch in test_data_local_dict[cid]]
        }

However, in the file fedml/data/fed_cifar100/data_loader.py, only 100 clients have a local test set while there are 500 clients with local training sets.

DEFAULT_TRAIN_CLIENTS_NUM = 500
DEFAULT_TEST_CLIENTS_NUM = 100

This causes all dataloaders for client IDs 200 to 500 in the test_data_local_dict dictionary to be None, making them unsuitable as iterators in the list comprehension.

Here is a temporary solution, but a more formal fix may be necessary:

tmp = {0 : []}
for cid in sorted(test_data_local_dict.keys()):
    if (test_data_local_dict[cid] != None):
        for batch in test_data_local_dict[cid]:
            tmp[0].append(batch)
test_data_local_dict = tmp

Guida contributor