FedML-AI/FedML

A bug encountered when using fed_cifar100 in centralized settings.

Open

#805 创建于 2023年3月6日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Python (743 fork)batch import
help wanted

仓库指标

Star
 (3,930 star)
PR 合并指标
 (30 天内没有已合并 PR)

描述

When using fed_cifar100 in centralized settings, I encountered a bug. In the file fedml/data/data_loader.py, line 559 constructs the test_data_local_dict in the following way:

test_data_local_dict = {
            0: [batch for cid in sorted(test_data_local_dict.keys()) for batch in test_data_local_dict[cid]]
        }

However, in the file fedml/data/fed_cifar100/data_loader.py, only 100 clients have a local test set while there are 500 clients with local training sets.

DEFAULT_TRAIN_CLIENTS_NUM = 500
DEFAULT_TEST_CLIENTS_NUM = 100

This causes all dataloaders for client IDs 200 to 500 in the test_data_local_dict dictionary to be None, making them unsuitable as iterators in the list comprehension.

Here is a temporary solution, but a more formal fix may be necessary:

tmp = {0 : []}
for cid in sorted(test_data_local_dict.keys()):
    if (test_data_local_dict[cid] != None):
        for batch in test_data_local_dict[cid]:
            tmp[0].append(batch)
test_data_local_dict = tmp

贡献者指南