Problem uploading empty files with upload_object
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
回報的進入點是 upload_object 和 upload_object_via_stream；先使用 S3 驅動重現空檔案的情況，並比較 AWS S3、MinIO 和 NetApp StorageGRID 之間的行為。由於 issue 中沒有指定原始檔案或測試，請先找到上傳實作及其測試。完成條件是 upload_object 能處理帶有中繼資料的空檔案，而不會產生重複的 Host 標頭，也不會破壞提供者相容性。
由索引模型根據 Issue 內容生成。
描述
Summary
Using upload_object to upload an empty file returns response:
400 - Bad Request: too many Host headers
Detailed Information
Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9
I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. upload_object function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
status: 400
reason: 'Bad Request: too many Host headers'
If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using upload_object_via_stream and found the following code works:
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).
I think using upload_object will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!
Reproduce
touch ~/test-empty.txt- Use
upload_objectto upload test-empty.txt
- 主要語言
- Python
- 星號
- 2.1k
- 分支
- 935
- 平均合併
- 5 天 3 小時
- 30 天內合併 PR
- 15
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
apache/libcloud 的其他 Issue
-
難度 2/5 1-3 小時 新手友好度 76/100
維護者通常 1 天內回覆
-
難度 1/5 1-3 小時 新手友好度 88/100
維護者通常 1 天內回覆
-
Local range download leaves an output file after source-read failure despite delete_on_failure=True未關閉
難度 2/5 1-3 小時 新手友好度 82/100
維護者通常 1 天內回覆
-
難度 5/5 一週以上 新手友好度 15/100
apache/libcloud#2215 · 3 則留言 · 2 個 reaction ·
維護者通常 1 天內回覆
-
難度 5/5 一週以上 新手友好度 35/100
維護者通常 1 天內回覆
相似的 Issue
-
Email validation accepts a trailing newline可能已有人在做 @marcello-pichini 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 72/100
dbt-labs/spark-utils#51 ·
-
難度 2/5 1-3 小時 新手友好度 85/100
維護者通常 1 天內回覆
-
bug frontend
難度 2/5 1-3 小時 新手友好度 88/100
vendrabuck/HERD#1419 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆