facebookresearch/fairscale
[chore] Cache wikitext2 dataset used for benchmarks/pipe.py to prevent throttling errors
オープン
#743 opened on 2021/07/19
good first issue
Repository metrics
- Stars
- (3,411 個のスター)
- PR merge metrics
- (PR metrics pending)
説明
Chore
We should cache the dataset used for running fairscale/benchmarks/pipe.py so that we don't end up downloading this at every run. This has been done for OSS and we should do something similar for pipe.py. This is important because during periods of high PR checkins/development we could run into throttling errors if there are too many CircleCI jobs. That is something that possibly happened for OSS which is when we added the caching logic.