Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Array(Nested(...)) columns silently decode as Array(Nothing) and desynchronize the native block stream

Aberta
  • Algo errado?
#571 1 comentário 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

@Hakaou já está trabalhando nisso.

Desde 17/10/2026.

  • #575 de @Hakaou — aberto
  • #590 de @Hakaou — aberto

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
68/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
clickhouse, cpp
Domínio
backend-api-design, database

Direção de pesquisa

Start with clickhouse/types/type_parser.cpp, tracing GetTypeMeta(), ValidateAST(), and CreateColumnFromAst for the Array(Nested(...)) reproduction. Then inspect clickhouse/columns/factory.cpp and the ColumnNothing behavior described in columns/nothing.h. Add coverage for the expected nested type or explicit unsupported-type failure, and verify the pure client-side test no longer yields Array(Nothing).

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Description

The type parser has no entry for Nested, so a column whose server-reported type is Array(Nested(key String, value String)) is parsed as Array(<unknown terminal>) with Type::Void, and CreateTerminalColumn() maps Type::Void to ColumnNothing. The result is that CreateColumnByType() returns a perfectly valid-looking ColumnArray(ColumnNothing) — no error, no nullptr — and Client::Select() happily accepts it when reading the block header.

This matters because Nested inside Array(...) is not flattened by the server. Unlike a top-level Nested column (which the server splits into col.key Array(String), col.value Array(String) — the case discussed in #40), Array(Nested(...)) is reported over the native protocol with the literal type name:

$ clickhouse-client -q "SELECT name, type FROM system.columns WHERE table='some_data'"
id      UInt64
data    Array(Nested(key String, value String))

On the wire Array(Nested(k String, v String)) is serialized as Array(Array(Tuple(String, String))). But the client builds Array(Nothing), and ColumnNothing::LoadBody() (clickhouse/columns/nothing.h:61) just does input->Skip(rows) — 1 byte per element. So after reading the outer offsets the client skips N bytes where the server actually wrote 8*N inner-offset bytes plus all the string data. The input stream is desynchronized from that point on: the remaining block (and every block after it) is decoded as garbage, typically surfacing much later as an unrelated parse failure or a corrupted/empty result rather than a clear "unsupported type" error.

This is the C++ analogue of ClickHouse/clickhouse-java#3178, where the JDBC driver also failed on Array(Nested(...)) because its conversion layer did not account for Nested producing an extra list level.

Related but distinct from #40, which asks for a convenience API over flattened top-level Nested columns (those already work, since the server hands them over as plain Array(T)). Here the client receives a literal Nested(...) type name and silently produces wrong data.

ClickHouse server version

26.9.8.3 — used to confirm the type name the server reports for Array(Nested(...)) (shown above).

Code analysis only; the C++ repro below was written but not executed — binary execution was unavailable in the environment I investigated from. The static trace through TypeParser::Parse → CreateColumnFromAst → CreateTerminalColumn is given in full under "Suggested fix" below, and the first assertion (Array(Nothing)) is a pure parser/factory fact requiring no server.

Reproduction

Pure client-side, no server needed — this already demonstrates the root cause:

#include <clickhouse/columns/factory.h>
#include <gtest/gtest.h>

using namespace clickhouse;

TEST(CreateColumnByType, ArrayOfNested) {
    auto col = CreateColumnByType("Array(Nested(key String, value String))");
    ASSERT_NE(nullptr, col);
    // Expected: Array(Array(Tuple(String, String)))
    // Actual:   Array(Nothing)
    EXPECT_EQ("Array(Array(Tuple(String, String)))", col->Type()->GetName());
}

End-to-end, against a server:

CREATE TABLE some_data (
    `id`   UInt64,
    `data` Array(Nested(`key` String, `value` String))
) ENGINE = Memory;

INSERT INTO some_data VALUES
(1, [[('key1','test'), ('key2','another-test')], [('key1','more-data')]]);

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions().SetHost("localhost").SetPort(9000));
    client.Select("SELECT id, data FROM some_data", [](const Block& block) {
        for (size_t c = 0; c < block.GetColumnCount(); ++c) {
            std::cout << block.GetColumnName(c) << " -> "
                      << block[c]->Type()->GetName() << "\n";
        }
    });
}

Expected: the data column comes back as Array(Array(Tuple(String, String))) with the two inner arrays intact (or, failing that, a clear UnimplementedError naming the unsupported type).

Actual: data is reported as Array(Nothing) carrying no values, and because ColumnNothing::LoadBody under-consumes the stream by the full size of the inner offsets and strings, decoding of the rest of the response is corrupted.

Suggested fix

Trace of the current behaviour:

  • clickhouse/types/type_parser.cpp:116 — GetTypeMeta() has no Nested branch, so the Nested(...) node falls through to TypeAst::Terminal.
  • clickhouse/types/type_parser.cpp:107 — GetTypeCode("Nested") misses kTypeCode and returns Type::Void.
  • clickhouse/types/type_parser.cpp:152 — ValidateAST() does reject unknown Terminal + Void nodes, but it is only ever called on the root AST node (type_parser.cpp:238). The root here is Array, so the bad child is never validated.
  • clickhouse/columns/factory.cpp:49 — CreateTerminalColumn() maps Type::Void to ColumnNothing, turning the unknown type into a silently-wrong column instead of a nullptr.

Two things worth doing, independently useful:

  1. Support Nested. Nested(a T1, b T2) is exactly Array(Tuple(a T1, b T2)). Adding a Nested meta that desugars to that in CreateColumnFromAst would make Array(Nested(...)) decode correctly, and would also let named-tuple element names (already supported via TypeAst::element_name) carry through.

  2. Fail loudly on unknown nested types. Apply ValidateAST recursively, or have CreateTerminalColumn return nullptr for Type::Void when ast.name is not literally Nothing/void. Right now any unrecognized type nested inside a container degrades to ColumnNothing and desynchronizes the stream rather than raising UnimplementedError. (Same silent-desync failure mode as #543.)

Link

Original client issue: https://github.com/ClickHouse/clickhouse-java/issues/3178

Linguagem predominante
C
Estrelas
386
Forks
212
Merge médio
18h 22min
PRs com merge (30d)
4

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de ClickHouse/clickhouse-cpp

Todas as issues de ClickHouse/clickhouse-cpp

Issues semelhantes

Mais issues de C

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.