[client-v2] Enum type parsing accepts duplicate ordinals and duplicate labels: Enum8('a' = 1, 'b' = 1) silently stores 'b' as 'a'

オープン 初心者向け
#3,349 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
76/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
java
領域
databases

調査の方向性

clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java から始めてください。Collection<String> コンストラクタ(37〜53行目付近)と (String[], int[]) コンストラクタ(55〜65行目付近)は、重複する順序値とラベルを受け入れています。両方のコンストラクタに、順序値またはラベルが重複したときに型名を含む IllegalArgumentException をスローするチェックを追加してください。ClickHouseColumn.parse が Enum8('a' = 1, 'b' = 1) と Enum8('a' = 1, 'a' = 2) を拒否し、Enum8('a' = 200, 'b' = 1) は引き続き解析できれば完了です。

索引モデルが issue の本文から書いたものです。

説明

Description

ClickHouseColumn.parse(...) / ClickHouseEnum accept an Enum type definition in which two members share an ordinal (Enum8('a' = 1, 'b' = 1)) or share a label (Enum8('a' = 1, 'a' = 2)). The server rejects both (Code: 62 ... Duplicate values in enum / Duplicate names in enum).

com.clickhouse.data.ClickHouseEnum stores the members in two parallel arrays and does no uniqueness check at all:

  • ClickHouseEnum(Collection<String> params) (clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java:37-53) only validates that each entry contains =.
  • value(String name) / name(int value) (lines 87-115) are first-match linear scans.

Consequence for a duplicate ordinal: value("a") == value("b") == 1, so both labels serialize to the same byte in SerializerUtils.serializeEnumData (client-v2/src/main/java/com/clickhouse/client/api/data_formats/internal/SerializerUtils.java:858-880), and name(1) only ever returns "a" — "b" is unreachable.

This is reachable whenever the application supplies the column type itself rather than letting the client probe the server: a TableSchema built from ClickHouseColumn.parse(...) and passed to Client.register(Class, TableSchema), then used for a RowBinary insert. The insert succeeds with no error and the wrong value lands in the table.

Impact is low — the server never emits such a type, so reads and server-probed insert schemas are unaffected. But the failure mode is silent data corruption rather than an error.

Found while checking whether ClickHouse/clickhouse-cs#731 also applies here. Distinct from #3304 (non-member integer / Enum.ordinal()).

ClickHouse server version

26.10.3.7 (verified against a running server; client built from main, version 0.12.0-rc1-SNAPSHOT).

The server rejects the type on every version:

SELECT CAST('a' AS Enum8('a' = 1, 'b' = 1))
Code: 62. DB::Exception: Duplicate values in enum: 'b' = 1 and 'a' ... (SYNTAX_ERROR)

Reproduction

client-v2/src/test/java/com/clickhouse/client/insert/EnumDupOrdinalTest.java:

package com.clickhouse.client.insert;

import com.clickhouse.client.api.Client;
import com.clickhouse.client.api.insert.InsertResponse;
import com.clickhouse.client.api.insert.InsertSettings;
import com.clickhouse.client.api.metadata.TableSchema;
import com.clickhouse.client.api.query.GenericRecord;
import com.clickhouse.data.ClickHouseColumn;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.List;

public class EnumDupOrdinalTest {

    public static class Row {
        private short id;
        private String e;

        public short getId() { return id; }
        public void setId(short id) { this.id = id; }
        public String getE() { return e; }
        public void setE(String e) { this.e = e; }
    }

    @Test
    public void duplicateOrdinalIsAccepted() throws Exception {
        ClickHouseColumn c = ClickHouseColumn.parse("e Enum8('a' = 1, 'b' = 1)").get(0);
        System.out.println("parsed OK (duplicate ordinal): " + c.getOriginalTypeName());
        System.out.println("  value('a') = " + c.getEnumConstants().value("a"));
        System.out.println("  value('b') = " + c.getEnumConstants().value("b"));
        System.out.println("  name(1)    = " + c.getEnumConstants().name(1));

        ClickHouseColumn c2 = ClickHouseColumn.parse("e Enum8('a' = 1, 'a' = 2)").get(0);
        System.out.println("parsed OK (duplicate label): " + c2.getOriginalTypeName());
        System.out.println("  value('a') = " + c2.getEnumConstants().value("a"));

        try (Client client = new Client.Builder().addEndpoint("http://localhost:8123")
                .setUsername("default").setPassword("").compressServerResponse(false).build()) {

            String table = "enum_dup_ordinal_test";
            client.execute("DROP TABLE IF EXISTS " + table).get();
            client.execute("CREATE TABLE " + table
                    + " (id UInt8, e Enum8('a' = 1, 'b' = 2)) ENGINE = Memory").get();

            // application-supplied schema, with a type the server would reject
            TableSchema schema = new TableSchema(table, null, "default",
                    ClickHouseColumn.parse("id UInt8, e Enum8('a' = 1, 'b' = 1)"));
            client.register(Row.class, schema);

            List<Object> data = new ArrayList<>();
            Row r1 = new Row(); r1.setId((short) 1); r1.setE("a");
            Row r2 = new Row(); r2.setId((short) 2); r2.setE("b");
            data.add(r1);
            data.add(r2);

            try (InsertResponse resp = client.insert(table, data, new InsertSettings()).get()) {
                System.out.println("insert succeeded, rows written = " + resp.getWrittenRows());
            }

            for (GenericRecord rec : client.queryAll(
                    "SELECT concat(toString(id), '|', toString(e), '|', toString(toInt16(e))) AS r FROM "
                            + table + " ORDER BY id")) {
                System.out.println("  row: " + rec.getString("r"));
            }
            client.execute("DROP TABLE IF EXISTS " + table).get();
        }
    }
}

Run: mvn -pl client-v2 -Dtest=EnumDupOrdinalTest test

Actual output (target/surefire-reports/...-output.txt):

parsed OK (duplicate ordinal): Enum8('a' = 1, 'b' = 1)
  value('a') = 1
  value('b') = 1
  name(1)    = a
parsed OK (duplicate label): Enum8('a' = 1, 'a' = 2)
  value('a') = 1
server version: 26.10.3.7
server rejected: SELECT CAST('a' AS Enum8('a' = 1, 'b' = 1))   <- server refuses the same type
insert succeeded, rows written = 2
  row: 1|a|1
  row: 2|a|1        <- 'b' was written as ordinal 1 and reads back as 'a'

Expected: ClickHouseColumn.parse / the ClickHouseEnum constructor reject both type strings with an exception naming the offending type, as the server does. The row written as 'b' should either fail or be stored as 'b'.

Suggested fix

In com.clickhouse.data.ClickHouseEnum (clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java), validate in the Collection<String> constructor (lines 37-53) and in the (String[], int[]) constructor (lines 55-65) that no ordinal and no label repeats, throwing IllegalArgumentException that names the type, e.g. Enum declares the ordinal 1 more than once / Enum declares the label 'a' more than once.

Note the contrast case that must keep working: an explicit out-of-range ordinal (Enum8('a' = 200, 'b' = 1)) is accepted and wrapped by older servers, so no range check should be added here.

Link

Relayed from https://github.com/ClickHouse/clickhouse-cs/issues/731
Tracking: https://github.com/ClickHouse/integrations-ai-playground/issues/693

主要言語
Java
スター
1.6k
フォーク
649
平均マージ
1日 20時間
マージ済み PR(30日)
39

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

ClickHouse/clickhouse-java のほかの issue

ClickHouse/clickhouse-java の issue をすべて見る

似ている issue

Java の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。