Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Array(Nested(...)) columns silently decode as Array(Nothing) and desynchronize the native block stream

Ouverte
  • Quelque chose ne va pas ?
#571 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@Hakaou y travaille déjà.

Depuis le 17/10/2026.

  • #575 par @Hakaou — ouverte
  • #590 par @Hakaou — ouverte

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
68/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
clickhouse, cpp
Domaine
backend-api-design, database

Piste de recherche

Start with clickhouse/types/type_parser.cpp, tracing GetTypeMeta(), ValidateAST(), and CreateColumnFromAst for the Array(Nested(...)) reproduction. Then inspect clickhouse/columns/factory.cpp and the ColumnNothing behavior described in columns/nothing.h. Add coverage for the expected nested type or explicit unsupported-type failure, and verify the pure client-side test no longer yields Array(Nothing).

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Description

The type parser has no entry for Nested, so a column whose server-reported type is Array(Nested(key String, value String)) is parsed as Array(<unknown terminal>) with Type::Void, and CreateTerminalColumn() maps Type::Void to ColumnNothing. The result is that CreateColumnByType() returns a perfectly valid-looking ColumnArray(ColumnNothing) — no error, no nullptr — and Client::Select() happily accepts it when reading the block header.

This matters because Nested inside Array(...) is not flattened by the server. Unlike a top-level Nested column (which the server splits into col.key Array(String), col.value Array(String) — the case discussed in #40), Array(Nested(...)) is reported over the native protocol with the literal type name:

$ clickhouse-client -q "SELECT name, type FROM system.columns WHERE table='some_data'"
id      UInt64
data    Array(Nested(key String, value String))

On the wire Array(Nested(k String, v String)) is serialized as Array(Array(Tuple(String, String))). But the client builds Array(Nothing), and ColumnNothing::LoadBody() (clickhouse/columns/nothing.h:61) just does input->Skip(rows) — 1 byte per element. So after reading the outer offsets the client skips N bytes where the server actually wrote 8*N inner-offset bytes plus all the string data. The input stream is desynchronized from that point on: the remaining block (and every block after it) is decoded as garbage, typically surfacing much later as an unrelated parse failure or a corrupted/empty result rather than a clear "unsupported type" error.

This is the C++ analogue of ClickHouse/clickhouse-java#3178, where the JDBC driver also failed on Array(Nested(...)) because its conversion layer did not account for Nested producing an extra list level.

Related but distinct from #40, which asks for a convenience API over flattened top-level Nested columns (those already work, since the server hands them over as plain Array(T)). Here the client receives a literal Nested(...) type name and silently produces wrong data.

ClickHouse server version

26.9.8.3 — used to confirm the type name the server reports for Array(Nested(...)) (shown above).

Code analysis only; the C++ repro below was written but not executed — binary execution was unavailable in the environment I investigated from. The static trace through TypeParser::Parse → CreateColumnFromAst → CreateTerminalColumn is given in full under "Suggested fix" below, and the first assertion (Array(Nothing)) is a pure parser/factory fact requiring no server.

Reproduction

Pure client-side, no server needed — this already demonstrates the root cause:

#include <clickhouse/columns/factory.h>
#include <gtest/gtest.h>

using namespace clickhouse;

TEST(CreateColumnByType, ArrayOfNested) {
    auto col = CreateColumnByType("Array(Nested(key String, value String))");
    ASSERT_NE(nullptr, col);
    // Expected: Array(Array(Tuple(String, String)))
    // Actual:   Array(Nothing)
    EXPECT_EQ("Array(Array(Tuple(String, String)))", col->Type()->GetName());
}

End-to-end, against a server:

CREATE TABLE some_data (
    `id`   UInt64,
    `data` Array(Nested(`key` String, `value` String))
) ENGINE = Memory;

INSERT INTO some_data VALUES
(1, [[('key1','test'), ('key2','another-test')], [('key1','more-data')]]);

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions().SetHost("localhost").SetPort(9000));
    client.Select("SELECT id, data FROM some_data", [](const Block& block) {
        for (size_t c = 0; c < block.GetColumnCount(); ++c) {
            std::cout << block.GetColumnName(c) << " -> "
                      << block[c]->Type()->GetName() << "\n";
        }
    });
}

Expected: the data column comes back as Array(Array(Tuple(String, String))) with the two inner arrays intact (or, failing that, a clear UnimplementedError naming the unsupported type).

Actual: data is reported as Array(Nothing) carrying no values, and because ColumnNothing::LoadBody under-consumes the stream by the full size of the inner offsets and strings, decoding of the rest of the response is corrupted.

Suggested fix

Trace of the current behaviour:

  • clickhouse/types/type_parser.cpp:116 — GetTypeMeta() has no Nested branch, so the Nested(...) node falls through to TypeAst::Terminal.
  • clickhouse/types/type_parser.cpp:107 — GetTypeCode("Nested") misses kTypeCode and returns Type::Void.
  • clickhouse/types/type_parser.cpp:152 — ValidateAST() does reject unknown Terminal + Void nodes, but it is only ever called on the root AST node (type_parser.cpp:238). The root here is Array, so the bad child is never validated.
  • clickhouse/columns/factory.cpp:49 — CreateTerminalColumn() maps Type::Void to ColumnNothing, turning the unknown type into a silently-wrong column instead of a nullptr.

Two things worth doing, independently useful:

  1. Support Nested. Nested(a T1, b T2) is exactly Array(Tuple(a T1, b T2)). Adding a Nested meta that desugars to that in CreateColumnFromAst would make Array(Nested(...)) decode correctly, and would also let named-tuple element names (already supported via TypeAst::element_name) carry through.

  2. Fail loudly on unknown nested types. Apply ValidateAST recursively, or have CreateTerminalColumn return nullptr for Type::Void when ast.name is not literally Nothing/void. Right now any unrecognized type nested inside a container degrades to ColumnNothing and desynchronizes the stream rather than raising UnimplementedError. (Same silent-desync failure mode as #543.)

Link

Original client issue: https://github.com/ClickHouse/clickhouse-java/issues/3178

Langage dominant
C
Étoiles
385
Forks
211
Merge moyen
18 h 22 min
PR mergées (30 j)
4

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de ClickHouse/clickhouse-cpp

Toutes les issues de ClickHouse/clickhouse-cpp

Issues similaires

Plus d'issues C

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.