Problem uploading empty files with upload_object
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
报告的入口点是 upload_object 和 upload_object_via_stream；首先使用 S3 驱动复现空文件情况，并比较 AWS S3、MinIO 和 NetApp StorageGRID 之间的行为。由于 issue 中没有指定源文件或测试，请先找到上传实现及其测试。完成标准是 upload_object 能够处理带有元数据的空文件，同时不会生成重复的 Host 标头，也不会破坏提供商兼容性。
由索引模型根据 Issue 内容生成。
描述
Summary
Using upload_object to upload an empty file returns response:
400 - Bad Request: too many Host headers
Detailed Information
Python 3.6.8
Libcloud: 3.3.1
OS: rhel 7.9
I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid, I use the s3 driver for all of them. upload_object function works fine except when I try to upload an empty file (I sometimes need to store just meta-data with a specific object name for later use). When trying to upload an empty file I get the response:
status: 400
reason: 'Bad Request: too many Host headers'
If I add anything to the file's content it uploads successfully with the meta-data, so its not a problem with permissions/path in s3 or something like that. I also tried without the meta-data, but that didn't work either.
I tried using upload_object_via_stream and found the following code works:
with io.BytesIO() as iterator:
driver.upload_object_via_stream(iterator=iterator,
container=container,
object_name=objName,
extra=extra)
This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's storageGrid (so unfortunately I can't use it).
I think it's a limitation of storageGrid's multi-part upload and not some other storage restriction (we have existing empty objects on storageGrid, but those were uploaded using jclouds).
I think using upload_object will be the best solution for all 3 providers, but I don't understand why it's not uploading empty files and would appreciate help with this. Thanks!
Reproduce
touch ~/test-empty.txt- Use
upload_objectto upload test-empty.txt
- 主要语言
- Python
- 星标
- 2.1k
- 派生
- 935
- 平均合并
- 5 天 3 小时
- 30 天内合并 PR
- 15
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/libcloud 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复
-
Local range download leaves an output file after source-read failure despite delete_on_failure=True未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 15/100
apache/libcloud#2215 · 3 条评论 · 2 个 reaction ·
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 35/100
维护者通常 1 天内回复
相似的 Issue
-
Email validation accepts a trailing newline可能已有人在做 @marcello-pichini 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
dbt-labs/spark-utils#51 ·
-
难度 2/5 1-3 小时 新手友好度 85/100
维护者通常 1 天内回复
-
bug frontend
难度 2/5 1-3 小时 新手友好度 88/100
vendrabuck/HERD#1419 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复