FedML-AI/FedML

A bug encountered when using fed_cifar100 in centralized settings.

Open

#805 opened on Mar 6, 2023

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Python (743 forks)batch import
help wanted

Repository metrics

Stars
 (3,930 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

When using fed_cifar100 in centralized settings, I encountered a bug. In the file fedml/data/data_loader.py, line 559 constructs the test_data_local_dict in the following way:

test_data_local_dict = {
            0: [batch for cid in sorted(test_data_local_dict.keys()) for batch in test_data_local_dict[cid]]
        }

However, in the file fedml/data/fed_cifar100/data_loader.py, only 100 clients have a local test set while there are 500 clients with local training sets.

DEFAULT_TRAIN_CLIENTS_NUM = 500
DEFAULT_TEST_CLIENTS_NUM = 100

This causes all dataloaders for client IDs 200 to 500 in the test_data_local_dict dictionary to be None, making them unsuitable as iterators in the list comprehension.

Here is a temporary solution, but a more formal fix may be necessary:

tmp = {0 : []}
for cid in sorted(test_data_local_dict.keys()):
    if (test_data_local_dict[cid] != None):
        for batch in test_data_local_dict[cid]:
            tmp[0].append(batch)
test_data_local_dict = tmp

Contributor guide

Research direction
Inspect the data loader logic for fed cifar100 to ensure test data is available for all clients or adjust the centralized data dict construction in `fedml/data/data loader.py` to handle None values.
Tech stack
python
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Python
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70