Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Array(Nested(...)) columns silently decode as Array(Nothing) and desynchronize the native block stream

Aperta
  • Qualcosa non va?
#571 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
68/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
clickhouse, cpp
Ambito
backend-api-design, database

Direzione di ricerca

Start with clickhouse/types/type_parser.cpp, tracing GetTypeMeta(), ValidateAST(), and CreateColumnFromAst for the Array(Nested(...)) reproduction. Then inspect clickhouse/columns/factory.cpp and the ColumnNothing behavior described in columns/nothing.h. Add coverage for the expected nested type or explicit unsupported-type failure, and verify the pure client-side test no longer yields Array(Nothing).

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description

The type parser has no entry for Nested, so a column whose server-reported type is Array(Nested(key String, value String)) is parsed as Array(<unknown terminal>) with Type::Void, and CreateTerminalColumn() maps Type::Void to ColumnNothing. The result is that CreateColumnByType() returns a perfectly valid-looking ColumnArray(ColumnNothing) — no error, no nullptr — and Client::Select() happily accepts it when reading the block header.

This matters because Nested inside Array(...) is not flattened by the server. Unlike a top-level Nested column (which the server splits into col.key Array(String), col.value Array(String) — the case discussed in #40), Array(Nested(...)) is reported over the native protocol with the literal type name:

$ clickhouse-client -q "SELECT name, type FROM system.columns WHERE table='some_data'"
id      UInt64
data    Array(Nested(key String, value String))

On the wire Array(Nested(k String, v String)) is serialized as Array(Array(Tuple(String, String))). But the client builds Array(Nothing), and ColumnNothing::LoadBody() (clickhouse/columns/nothing.h:61) just does input->Skip(rows) — 1 byte per element. So after reading the outer offsets the client skips N bytes where the server actually wrote 8*N inner-offset bytes plus all the string data. The input stream is desynchronized from that point on: the remaining block (and every block after it) is decoded as garbage, typically surfacing much later as an unrelated parse failure or a corrupted/empty result rather than a clear "unsupported type" error.

This is the C++ analogue of ClickHouse/clickhouse-java#3178, where the JDBC driver also failed on Array(Nested(...)) because its conversion layer did not account for Nested producing an extra list level.

Related but distinct from #40, which asks for a convenience API over flattened top-level Nested columns (those already work, since the server hands them over as plain Array(T)). Here the client receives a literal Nested(...) type name and silently produces wrong data.

ClickHouse server version

26.9.8.3 — used to confirm the type name the server reports for Array(Nested(...)) (shown above).

Code analysis only; the C++ repro below was written but not executed — binary execution was unavailable in the environment I investigated from. The static trace through TypeParser::Parse → CreateColumnFromAst → CreateTerminalColumn is given in full under "Suggested fix" below, and the first assertion (Array(Nothing)) is a pure parser/factory fact requiring no server.

Reproduction

Pure client-side, no server needed — this already demonstrates the root cause:

#include <clickhouse/columns/factory.h>
#include <gtest/gtest.h>

using namespace clickhouse;

TEST(CreateColumnByType, ArrayOfNested) {
    auto col = CreateColumnByType("Array(Nested(key String, value String))");
    ASSERT_NE(nullptr, col);
    // Expected: Array(Array(Tuple(String, String)))
    // Actual:   Array(Nothing)
    EXPECT_EQ("Array(Array(Tuple(String, String)))", col->Type()->GetName());
}

End-to-end, against a server:

CREATE TABLE some_data (
    `id`   UInt64,
    `data` Array(Nested(`key` String, `value` String))
) ENGINE = Memory;

INSERT INTO some_data VALUES
(1, [[('key1','test'), ('key2','another-test')], [('key1','more-data')]]);

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions().SetHost("localhost").SetPort(9000));
    client.Select("SELECT id, data FROM some_data", [](const Block& block) {
        for (size_t c = 0; c < block.GetColumnCount(); ++c) {
            std::cout << block.GetColumnName(c) << " -> "
                      << block[c]->Type()->GetName() << "\n";
        }
    });
}

Expected: the data column comes back as Array(Array(Tuple(String, String))) with the two inner arrays intact (or, failing that, a clear UnimplementedError naming the unsupported type).

Actual: data is reported as Array(Nothing) carrying no values, and because ColumnNothing::LoadBody under-consumes the stream by the full size of the inner offsets and strings, decoding of the rest of the response is corrupted.

Suggested fix

Trace of the current behaviour:

  • clickhouse/types/type_parser.cpp:116 — GetTypeMeta() has no Nested branch, so the Nested(...) node falls through to TypeAst::Terminal.
  • clickhouse/types/type_parser.cpp:107 — GetTypeCode("Nested") misses kTypeCode and returns Type::Void.
  • clickhouse/types/type_parser.cpp:152 — ValidateAST() does reject unknown Terminal + Void nodes, but it is only ever called on the root AST node (type_parser.cpp:238). The root here is Array, so the bad child is never validated.
  • clickhouse/columns/factory.cpp:49 — CreateTerminalColumn() maps Type::Void to ColumnNothing, turning the unknown type into a silently-wrong column instead of a nullptr.

Two things worth doing, independently useful:

  1. Support Nested. Nested(a T1, b T2) is exactly Array(Tuple(a T1, b T2)). Adding a Nested meta that desugars to that in CreateColumnFromAst would make Array(Nested(...)) decode correctly, and would also let named-tuple element names (already supported via TypeAst::element_name) carry through.

  2. Fail loudly on unknown nested types. Apply ValidateAST recursively, or have CreateTerminalColumn return nullptr for Type::Void when ast.name is not literally Nothing/void. Right now any unrecognized type nested inside a container degrades to ColumnNothing and desynchronizes the stream rather than raising UnimplementedError. (Same silent-desync failure mode as #543.)

Link

Original client issue: https://github.com/ClickHouse/clickhouse-java/issues/3178

Lingua principale
C
Stelle
383
Fork
209
Merge medio
2g 13h
PR unite (30g)
7

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ClickHouse/clickhouse-cpp

Tutte le issue di ClickHouse/clickhouse-cpp

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.