skypilot-org/skypilot
[GCP] Move from `gsutil cp/rsync` to `gcloud storage cp/rsync`
開放
#8,457 建立於 2025年12月30日
good first issuegood starter issues
倉庫指標
- 星標
- (4,859 顆星)
- PR 合併指標
- (平均合併 2天 23小時) (30 天內合併 146 個 PR)
描述
According to https://docs.cloud.google.com/storage/docs/working-with-big-data, gcloud storage cp is much faster due to optimizations in automatic parallelism. We can migrate the usage gsutil cp in https://github.com/skypilot-org/skypilot/blob/e6a41b121ae27c33e1815ecf316858752dcc9982/sky/data/storage.py#L2370.
I manually tried a few file downloads with gcloud storage cp, it easily saturates the 300MB/s throughput, i.e. it worth a try.