Native TCP: CompressionMethod::ZSTD is not honored for server -> client blocks (network_compression_method never sent)

Open
#556 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
68/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp
Domain
api, networking

Research direction

Start with ClientOptions in clickhouse/client.h and the compression handling in clickhouse/client.cpp:349-351 and 1067-1080. Trace Client::Impl::SendQuery and the native TCP query settings, then use the supplied reproduction to verify that ZSTD sends network_compression_method and that server-to-client blocks use the requested codec; preserve existing settings when already present.

Written by the indexing model from the issue text.

Description

Description

When a client is configured with ClientOptions().SetCompressionMethod(CompressionMethod::ZSTD), only the client → server direction actually uses ZSTD. The server → client direction still uses the server's default codec (LZ4).

The reason is that the native TCP query packet only carries a boolean "compression enabled" flag; the selected codec is never propagated to the server:

  • clickhouse/client.cpp:349-351compression_ is set to CompressionState::Enable for any method other than None; the specific method is discarded.
  • clickhouse/client.cpp:1087WireFormat::WriteUInt64(*output_, compression_); writes only that enable flag.
  • clickhouse/client.cpp:447CompressedOutput correctly uses options_.compression_method for the outgoing (client → server) data blocks.

To make the server compress its responses with ZSTD, the client must send the network_compression_method (and, for a configured level, network_zstd_compression_level) setting in the per-query settings section of the query packet (clickhouse/client.cpp:1067-1080). It never does.

The read path is not broken — CompressedInput::Decompress (clickhouse/base/compressed.cpp:105-131) accepts both LZ4 and ZSTD method bytes — so nothing fails; the selected codec is just silently not honored in one direction.

Additionally, there is no option to configure a ZSTD compression level at all: ClientOptions (clickhouse/client.h:107) only exposes compression_method.

This is the C++ equivalent of ClickHouse/clickhouse-go#1993.

ClickHouse server version

26.7.5.10 (verified against a running server over the native protocol on port 9000).

Reproduction

The server-side effective value of network_compression_method for the query is what determines how the server compresses the result blocks it sends back, so it can be observed directly from the client:

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions()
                      .SetHost("localhost")
                      .SetPort(9000)
                      .SetCompressionMethod(CompressionMethod::ZSTD));

    client.Select(
        "SELECT name, value FROM system.settings WHERE name IN "
        "('network_compression_method', 'network_zstd_compression_level')",
        [](const Block& block) {
            for (size_t i = 0; i < block.GetRowCount(); ++i) {
                std::cout << (*block[0]->As<ColumnString>())[i] << " = "
                          << (*block[1]->As<ColumnString>())[i] << std::endl;
            }
        });
    return 0;
}

Expected output (codec honored in both directions):

network_compression_method = ZSTD
network_zstd_compression_level = <configured level>

Actual output:

network_compression_method = LZ4
network_zstd_compression_level = 1

i.e. the server compresses everything it sends back to this client with LZ4, even though ZSTD was requested.

Suggested fix

In Client::Impl::SendQuery (clickhouse/client.cpp, per-query settings block around lines 1067-1080), when options_.compression_method != CompressionMethod::None and the setting is not already present in query.GetQuerySettings(), send:

  • network_compression_method = "ZSTD" / "LZ4" matching options_.compression_method
  • network_zstd_compression_level = the configured level, if a new ClientOptions field for it is added (e.g. SetCompressionLevel), when the method is ZSTD

Link

Original report against the Go client: https://github.com/ClickHouse/clickhouse-go/issues/1993

Dominant language
C
Stars
382
Forks
208
Avg merge
2d 19h
Merged PRs (30d)
14

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ClickHouse/clickhouse-cpp

All issues in ClickHouse/clickhouse-cpp

Similar issues

More C issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.