ClickHouse/ClickHouse

gcs : The request signature we calculated does not match the signature you provided

Ouverte

#37 939 ouverte le 9 juin 2022

 (7 commentaires) (0 réaction) (0 personne assignée)C++ (8 400 forks)batch import
help wantedunexpected behaviour

Métriques du dépôt

Stars
 (47 419 étoiles)
Métriques de merge PR
 (Merge moyen 2j 2h) (1 000 PRs mergées en 30 j)

Description

Similar to https://github.com/ClickHouse/ClickHouse/issues/33009 & https://github.com/ClickHouse/ClickHouse/pull/37344, but I not using s3 but url (I signed it via python and passed to clickhouse), can we perform the same action to all the request's URI passed to sendRequest as https://github.com/ClickHouse/ClickHouse/pull/37344 did?

src/Storages/MeiliSearch/MeiliSearchConnection.cpp:40:    std::ostream & os = session.sendRequest(req);
src/Storages/MeiliSearch/MeiliSearchConnection.cpp:75:    session.sendRequest(req);
src/IO/WriteBufferFromHTTP.cpp:33:    ostr = &session->sendRequest(request);
src/IO/S3/PocoHTTPClient.cpp:253:            auto & request_body_stream = session->sendRequest(poco_request);
src/IO/ReadWriteBufferFromHTTP.h:182:                auto & stream_out = sess->sendRequest(request);
src/Processors/Formats/Impl/AvroRowInputFormat.cpp:680:                    session->sendRequest(request);
src/Processors/Transforms/MongoDBSource.cpp:58:        connection.sendRequest(*command, response);
src/Processors/Transforms/MongoDBSource.cpp:113:        connection.sendRequest(*command, response);
src/Disks/ObjectStorages/S3/ProxyResolverConfiguration.cpp:70:                session->sendRequest(request);

Another question: Would you recommend using url or s3 for downloading files from s3/gcs in regard to performance? I found they will both perform ranged download, will s3 first do some bucket list? I also found that s3 supports compression while the url doesn't (although it seems that URL table engine does support)

Guide contributeur