cube-js/cube

Cubestore s3 fails when uploading with`File can't be listed after upload.` when subPath starts with /

Open

#6,136 建立於 2023年2月8日

在 GitHub 查看
 (1 留言) (1 反應) (0 負責人)Rust (1,965 fork)batch import
cube storehelp wanted

倉庫指標

Star
 (19,563 star)
PR 合併指標
 (平均合併 5天 16小時) (30 天內合併 138 個 PR)

描述

Describe the bug Errors like:

Error during Metastore upload: CubeError { message: "File metastore-1675803621852-logs/2.flex can't be listed after upload. Either there's Cube Store cluster misconfiguration, or storage can't provide the required consistency.", backtrace: "", cause: Internal }

for every uploaded file to s3 if subpath starts with /

To Reproduce Steps to reproduce the behavior: Deploy cubestore with AWS s3 storage and use a subpath with leading /

Expected behavior Error shouldnt happen as AWS now provides strong read-after-write consistency

Screenshots Check related slack thread

Minimally reproducible Cube Schema

Version: master

Additional context Related rust-s3 open issue: https://github.com/durch/rust-s3/issues/203

I fixed it by removing the leading / on my subpath, but cubestore should be able to manage this case silently.

貢獻者指南