advancedenhancementhelp wanted
仓库指标
- Star
- (3,775 star)
- PR 合并指标
- (PR 指标待抓取)
描述
We do some proper distributed tests in tests/utils/test_dist__gather__losses.py, however those are separate from the test modules of the individual losses so adding new losses always gets forgotten or we try to mock the distributed behaviour, which often does not work very well, e.g. in #1980 .
Instead, it would be nice to have a shared distributed context that we could submit distributed tests to. This should run entirely on CPU (gloo backend for PyTorch). A really nice example for this is torchmetrics.
Let me know if you'd like to work on this and I will give some more proper guidance and detail on how this can be done.
Following things to be done:
- add distributed pool to the test setup following torchmetrics (test with a single loss)
- add to CI
- abstract it nicely
- expand coverage to other losses/use cases
- get rid of
tests/utils/test_dist__gather__losses.py