drivendataorg/cloudpathlib

MinIO custom S3 endpoint does not support our `touch` implementation

オープン

#141 opened on 2021/04/03

 (0 件のコメント) (0 件のリアクション) (0 人の担当者)Python (87 件のフォーク)auto 404
S3bughelp wanted

Repository metrics

Stars
 (624 個のスター)
PR merge metrics
 (PR metrics pending)

説明

Our touch implementation depends on the MetadataDirective="REPLACE": https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/cloudpathlib/s3/s3client.py#L158-L165

However, this fails our test for touch when the rig is custom_s3_rig running against our MinIO test server: https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/tests/test_cloudpath_file_io.py#L61-L66

I suspect that MinIO doesn't support this S3 functionality yet or it is a bug. We potentially could workaround by checking if the endpoint_url is set to a non-AWS URL and potentially do the fallback entire copy in that case.

This needs research to understand if this is a MinIO bug we could file to be fixed upstream or if we should workaround it.

When this is fixed, we should re-enable this test.

コントリビューターガイド