Native TCP: CompressionMethod::ZSTD is not honored for server -> client blocks (network_compression_method never sent)

Aperta
#556 0 commenti 1 reazione 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
68/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
api, networking

Direzione di ricerca

Inizia da ClientOptions in clickhouse/client.h e dalla gestione della compressione in clickhouse/client.cpp:349-351 e 1067-1080. Traccia Client::Impl::SendQuery e le impostazioni delle query TCP native, quindi usa la riproduzione fornita per verificare che ZSTD invii network_compression_method e che i blocchi dal server al client utilizzino il codec richiesto; preserva le impostazioni esistenti quando sono già presenti.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description

When a client is configured with ClientOptions().SetCompressionMethod(CompressionMethod::ZSTD), only the client → server direction actually uses ZSTD. The server → client direction still uses the server's default codec (LZ4).

The reason is that the native TCP query packet only carries a boolean "compression enabled" flag; the selected codec is never propagated to the server:

  • clickhouse/client.cpp:349-351compression_ is set to CompressionState::Enable for any method other than None; the specific method is discarded.
  • clickhouse/client.cpp:1087WireFormat::WriteUInt64(*output_, compression_); writes only that enable flag.
  • clickhouse/client.cpp:447CompressedOutput correctly uses options_.compression_method for the outgoing (client → server) data blocks.

To make the server compress its responses with ZSTD, the client must send the network_compression_method (and, for a configured level, network_zstd_compression_level) setting in the per-query settings section of the query packet (clickhouse/client.cpp:1067-1080). It never does.

The read path is not broken — CompressedInput::Decompress (clickhouse/base/compressed.cpp:105-131) accepts both LZ4 and ZSTD method bytes — so nothing fails; the selected codec is just silently not honored in one direction.

Additionally, there is no option to configure a ZSTD compression level at all: ClientOptions (clickhouse/client.h:107) only exposes compression_method.

This is the C++ equivalent of ClickHouse/clickhouse-go#1993.

ClickHouse server version

26.7.5.10 (verified against a running server over the native protocol on port 9000).

Reproduction

The server-side effective value of network_compression_method for the query is what determines how the server compresses the result blocks it sends back, so it can be observed directly from the client:

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions()
                      .SetHost("localhost")
                      .SetPort(9000)
                      .SetCompressionMethod(CompressionMethod::ZSTD));

    client.Select(
        "SELECT name, value FROM system.settings WHERE name IN "
        "('network_compression_method', 'network_zstd_compression_level')",
        [](const Block& block) {
            for (size_t i = 0; i < block.GetRowCount(); ++i) {
                std::cout << (*block[0]->As<ColumnString>())[i] << " = "
                          << (*block[1]->As<ColumnString>())[i] << std::endl;
            }
        });
    return 0;
}

Expected output (codec honored in both directions):

network_compression_method = ZSTD
network_zstd_compression_level = <configured level>

Actual output:

network_compression_method = LZ4
network_zstd_compression_level = 1

i.e. the server compresses everything it sends back to this client with LZ4, even though ZSTD was requested.

Suggested fix

In Client::Impl::SendQuery (clickhouse/client.cpp, per-query settings block around lines 1067-1080), when options_.compression_method != CompressionMethod::None and the setting is not already present in query.GetQuerySettings(), send:

  • network_compression_method = "ZSTD" / "LZ4" matching options_.compression_method
  • network_zstd_compression_level = the configured level, if a new ClientOptions field for it is added (e.g. SetCompressionLevel), when the method is ZSTD

Link

Original report against the Go client: https://github.com/ClickHouse/clickhouse-go/issues/1993

Lingua principale
C
Stelle
382
Fork
208
Merge medio
2g 19h
PR unite (30g)
14

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ClickHouse/clickhouse-cpp

Tutte le issue di ClickHouse/clickhouse-cpp

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.