Native TCP: CompressionMethod::ZSTD is not honored for server -> client blocks (network_compression_method never sent)
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 68/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- cpp
- Domain
- api, networking
Research direction
Start with ClientOptions in clickhouse/client.h and the compression handling in clickhouse/client.cpp:349-351 and 1067-1080. Trace Client::Impl::SendQuery and the native TCP query settings, then use the supplied reproduction to verify that ZSTD sends network_compression_method and that server-to-client blocks use the requested codec; preserve existing settings when already present.
Written by the indexing model from the issue text.
Description
Description
When a client is configured with ClientOptions().SetCompressionMethod(CompressionMethod::ZSTD), only the client → server direction actually uses ZSTD. The server → client direction still uses the server's default codec (LZ4).
The reason is that the native TCP query packet only carries a boolean "compression enabled" flag; the selected codec is never propagated to the server:
clickhouse/client.cpp:349-351—compression_is set toCompressionState::Enablefor any method other thanNone; the specific method is discarded.clickhouse/client.cpp:1087—WireFormat::WriteUInt64(*output_, compression_);writes only that enable flag.clickhouse/client.cpp:447—CompressedOutputcorrectly usesoptions_.compression_methodfor the outgoing (client → server) data blocks.
To make the server compress its responses with ZSTD, the client must send the network_compression_method (and, for a configured level, network_zstd_compression_level) setting in the per-query settings section of the query packet (clickhouse/client.cpp:1067-1080). It never does.
The read path is not broken — CompressedInput::Decompress (clickhouse/base/compressed.cpp:105-131) accepts both LZ4 and ZSTD method bytes — so nothing fails; the selected codec is just silently not honored in one direction.
Additionally, there is no option to configure a ZSTD compression level at all: ClientOptions (clickhouse/client.h:107) only exposes compression_method.
This is the C++ equivalent of ClickHouse/clickhouse-go#1993.
ClickHouse server version
26.7.5.10 (verified against a running server over the native protocol on port 9000).
Reproduction
The server-side effective value of network_compression_method for the query is what determines how the server compresses the result blocks it sends back, so it can be observed directly from the client:
#include <clickhouse/client.h>
#include <iostream>
using namespace clickhouse;
int main() {
Client client(ClientOptions()
.SetHost("localhost")
.SetPort(9000)
.SetCompressionMethod(CompressionMethod::ZSTD));
client.Select(
"SELECT name, value FROM system.settings WHERE name IN "
"('network_compression_method', 'network_zstd_compression_level')",
[](const Block& block) {
for (size_t i = 0; i < block.GetRowCount(); ++i) {
std::cout << (*block[0]->As<ColumnString>())[i] << " = "
<< (*block[1]->As<ColumnString>())[i] << std::endl;
}
});
return 0;
}
Expected output (codec honored in both directions):
network_compression_method = ZSTD
network_zstd_compression_level = <configured level>
Actual output:
network_compression_method = LZ4
network_zstd_compression_level = 1
i.e. the server compresses everything it sends back to this client with LZ4, even though ZSTD was requested.
Suggested fix
In Client::Impl::SendQuery (clickhouse/client.cpp, per-query settings block around lines 1067-1080), when options_.compression_method != CompressionMethod::None and the setting is not already present in query.GetQuerySettings(), send:
network_compression_method="ZSTD"/"LZ4"matchingoptions_.compression_methodnetwork_zstd_compression_level= the configured level, if a newClientOptionsfield for it is added (e.g.SetCompressionLevel), when the method is ZSTD
Link
Original report against the Go client: https://github.com/ClickHouse/clickhouse-go/issues/1993
- Dominant language
- C
- Stars
- 382
- Forks
- 208
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 14
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ClickHouse/clickhouse-cpp
-
enhancement pg_clickhouse
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
ClickHouse/clickhouse-cpp#478 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
ClickHouse/clickhouse-cpp#565 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
ClickHouse/clickhouse-cpp#560 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
ClickHouse/clickhouse-cpp#549 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 75/100
ClickHouse/clickhouse-cpp#543 ·
All issues in ClickHouse/clickhouse-cpp
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
BasedHardware/omi#15662 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
level/task module/gcp type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
hapostgres/pg_auto_failover#1190 ·