drivendataorg/cloudpathlib

MinIO custom S3 endpoint does not support our `touch` implementation

Aberta

#141 aberto em 3 de abr. de 2021

 (0 comentário) (0 reação) (0 responsável)Python (87 forks)auto 404
S3bughelp wanted

Métricas do repositório

Stars
 (624 estrelas)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

Our touch implementation depends on the MetadataDirective="REPLACE": https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/cloudpathlib/s3/s3client.py#L158-L165

However, this fails our test for touch when the rig is custom_s3_rig running against our MinIO test server: https://github.com/drivendataorg/cloudpathlib/blob/de6b54737014b06982e8c3af54d2da4238fcd8e3/tests/test_cloudpath_file_io.py#L61-L66

I suspect that MinIO doesn't support this S3 functionality yet or it is a bug. We potentially could workaround by checking if the endpoint_url is set to a non-AWS URL and potentially do the fallback entire copy in that case.

This needs research to understand if this is a MinIO bug we could file to be fixed upstream or if we should workaround it.

When this is fixed, we should re-enable this test.

Guia do colaborador