BUG when Trainer.test() with deepspeed stage 3
#14 090 ouverte le 8 août 2022
Métriques du dépôt
- Stars
- (26 687 stars)
- Métriques de merge PR
- (Merge moyen 9j 15h) (3 PRs mergées en 30 j)
Description
🐛 Bug
Hi, I meet some bugs when I combine lightning with deepspeed follow https://pytorch-lightning.readthedocs.io/en/latest/common/trainer.html#trainer-class-api.
I find that:
- Trainer.fit() is compatible well with deepspeed stage 2 and 3 .
- Trainer.test() is compatible with deepspeed stage 2.
- There is some bugs when combine Trainer.test() with deepspeed stage 3:
When initialize DeepSpeedStrategy with deepspeed config and then test, I will encounter this bug, I have tried to change the config, but this bug is exsited whatever.
When initialize DeepSpeedStrategy with lightning args, I will encounter two problems, the generate speed is too slow (I used huggingface transformers and their generate() method) than the deepspeed inference scripts in https://www.deepspeed.ai/tutorials/inference-tutorial/. In addition, I also meet a bug called index of range because of self.__step of deepspeed engine.
Hope for better compatibility between lightning and deepspeed !
cc @awaelchli @rohitgr7 @akihironitta