[client-v2] Enum type name without explicit values (Enum8('a', 'b')) fails to parse: Invalid enum entry

Offen
#3,344 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
Ein halber Tag
Anfängerfreundlichkeit
55/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
java
Bereich
databases

Rechercherichtung

Start in clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java, constructor at lines 37-53, which is called from ClickHouseColumn.update() at line 181. Read the existing negative cases in ClickHouseColumnTest.testEnum before changing the parsing. Done when ClickHouseColumn.of("e", "Enum8('a', 'b')") yields the same constants as Enum8('a' = 1, 'b' = 2), names containing '=' parse correctly, and unquoted or mixed lists are still rejected.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

area:data-type bug client-api-v2 jdbc-v2

Description

The server accepts Enum elements without a number and assigns the numbers itself. The client type-name parser rejects them. ClickHouseColumn.of("e", "Enum8('a', 'b')") throws IllegalArgumentException: Invalid enum entry: 'a'. The same occurs for Enum(...), Enum16(...), for a list with a start value (Enum8('a' = 5, 'b')), and when the type is inside Nullable, Array, Tuple or Map.

Server docs (Enum): "you could omit numbers. ClickHouse will assign consecutive numbers automatically. Numbers are assigned starting from 1 by default." and "You can also specify legal starting number for the first name."

An element name that contains = and has no number gives a different error: Enum8('x=y', 'b') throws NumberFormatException: For input string: "y'".

Where it is reachable: only type names that the application writes:

  • client-v2: ClickHouseColumn.of(...) in a TableSchema for Client.newBinaryFormatReader(response, schema) (RowBinary without a header), RowBinaryFormatWriter, or register(Class, schema).
  • jdbc-v2: Connection.createArrayOf(typeName, …) and Connection.createStruct("Tuple(…)", …).

Type names from the server (RowBinaryWithNamesAndTypes / Native header, getTableSchema) always have explicit numbers, so they are not affected. The failure is an exception, not wrong data.

Root cause

ClickHouseEnum(Collection<String>) (clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java:37-53, called from ClickHouseColumn.update() at ClickHouseColumn.java:181) requires the form name=value for each element. It splits on p.lastIndexOf('=') and throws Invalid enum entry when there is no = (line 49). For a name that contains =, lastIndexOf finds the = inside the quoted name, and Integer.parseInt fails on the rest of the name (line 47).

Suggested fix

In ClickHouseEnum(Collection<String>), read each element as a quoted name (with '' and \' escapes), then an optional = <integer>. Do not use lastIndexOf('='). Assign the numbers as the server does (autoAssignNumberForEnum in src/DataTypes/DataTypeEnum.cpp):

  • an element without a number at position i gets base + i. base is 1, or the number of the first element if the first element has a number;
  • the server accepts a mixed list only if all elements after the first have no number, or all elements after the first have a number.

Behavior that must not change (the server rejects these too):

  • unquoted names, for example Enum8(aa, 1) (existing negative case in ClickHouseColumnTest.testEnum). Server: Code: 223 … Elements of Enum data type must be of form: 'name' = number or 'name';
  • other mixed lists, for example Enum8('a' = 5, 'b', 'c' = 10, 'd'). Server: Code: 223 … All elements of Enum data type must be of form ….

Related: #3342 (width of a bare Enum(...)). If the width of a bare Enum comes from the element numbers, assign the implicit numbers first: Enum('a' = 127, 'b') is Enum16('a' = 127, 'b' = 128) on the server.

Steps to reproduce
  1. Build main (f73e372bc).
  2. Call ClickHouseColumn.of("e", "Enum8('a', 'b')"), or use that type name in a TableSchema for newBinaryFormatReader / RowBinaryFormatWriter.
  3. The call throws IllegalArgumentException: Invalid enum entry: 'a'. With Enum8('a' = 1, 'b' = 2) the same code works.
Error Log or Exception StackTrace
java.lang.IllegalArgumentException: Invalid enum entry: 'a'
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:49)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)
	at com.clickhouse.data.ClickHouseColumn.readColumn(ClickHouseColumn.java:732)
	at com.clickhouse.data.ClickHouseColumn.of(ClickHouseColumn.java:763)

java.lang.NumberFormatException: For input string: "y'"     (type Enum8('x=y', 'b'))
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:47)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)

Server type (SELECT toTypeName(defaultValueOfTypeName('<type>'))) compared with the client parser (ClickHouseColumn.of("e", "<type>")):

type                                server                                 client
Enum8('a' = 1, 'b' = 2)             Enum8('a' = 1, 'b' = 2)                Enum8 {a=1, b=2}
Enum('a', 'b')                      Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum8('a', 'b')                     Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum16('a', 'b')                    Enum16('a' = 1, 'b' = 2)               Invalid enum entry: 'a'
Enum8('a' = 5, 'b')                 Enum8('a' = 5, 'b' = 6)                Invalid enum entry: 'b'
Enum8('a', 'b' = 5)                 Enum8('a' = 1, 'b' = 5)                Invalid enum entry: 'a'
Enum16('a' = 1000, 'b')             Enum16('a' = 1000, 'b' = 1001)         Invalid enum entry: 'b'
Enum8('a' = -3, 'b', 'c')           Enum8('a' = -3, 'b' = -2, 'c' = -1)    Invalid enum entry: 'b'
Enum8('Query''Start', 'b')          Enum8('Query\'Start' = 1, 'b' = 2)     Invalid enum entry: 'Query''Start'
Enum8('x=y', 'b')                   Enum8('x=y' = 1, 'b' = 2)              NumberFormatException: "y'"
Nullable(Enum8('a', 'b'))           Nullable(Enum8('a' = 1, 'b' = 2))      Invalid enum entry: 'a'
Array(Enum8('a', 'b'))              Array(Enum8('a' = 1, 'b' = 2))         Invalid enum entry: 'a'
Tuple(Enum8('a', 'b'), UInt8)       Tuple(Enum8('a' = 1, 'b' = 2), UInt8)  Invalid enum entry: 'a'
Map(String, Enum8('a', 'b'))        Map(String, Enum8('a' = 1, 'b' = 2))   Invalid enum entry: 'a'
Enum8('a' = 5, 'b', 'c' = 10, 'd')  error (Code: 223)                      Invalid enum entry: 'b'   (both reject: OK)
Enum8(aa, 1)                        error (Code: 223)                      Invalid enum entry: aa    (both reject: OK)

End-to-end results (same program):

client-v2 read, RowBinary, schema e Enum8('a' = 1, 'b' = 2)  -> e=b u=7
client-v2 read, RowBinary, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
client-v2 RowBinaryFormatWriter, schema e Enum8('a' = 1, 'b' = 2) -> inserted OK
client-v2 RowBinaryFormatWriter, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createArrayOf("Enum8('a', 'b')", …)                  -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createStruct("Tuple(Enum8('a', 'b'), UInt8)", …)     -> IllegalArgumentException: Invalid enum entry: 'a'
Expected Behaviour

The client parses the type name with the numbers that the server assigns. Enum8('a', 'b') gives the same constants as Enum8('a' = 1, 'b' = 2), and the reader and writer work with it.

Code Example
// Parser
ClickHouseColumn.of("e", "Enum8('a', 'b')"); // throws IllegalArgumentException: Invalid enum entry: 'a'

// client-v2 read with an application schema
TableSchema schema = new TableSchema(Arrays.asList(
        ClickHouseColumn.of("e", "Enum8('a', 'b')"),   // throws here
        ClickHouseColumn.of("u", "UInt8")));
try (QueryResponse resp = client.query(
            "SELECT CAST('b', 'Enum8(\\'a\\' = 1, \\'b\\' = 2)') AS e, toUInt8(7) AS u",
            new QuerySettings().setFormat(ClickHouseFormat.RowBinary)).get();
     ClickHouseBinaryFormatReader r = client.newBinaryFormatReader(resp, schema)) {
    r.next();
    System.out.println(r.getString("e"));
}
Configuration
Client Configuration
Client client = new Client.Builder()
        .addEndpoint("http://<host>:8123")
        .setUsername("default").setPassword("<password>")
        .build();
Environment
  • Cloud
  • Client version: main at f73e372bc (0.12.0-rc1-SNAPSHOT): clickhouse-data, client-v2, jdbc-v2
  • Language version: OpenJDK 17.0.21
  • OS: Linux (Docker)
ClickHouse Server
  • ClickHouse Server version: 26.9.1.1629
  • ClickHouse Server non-default settings, if any: none
  • CREATE TABLE statements for tables involved: CREATE TABLE enum_impl_w (e Enum8('a', 'b'), u UInt8) ENGINE = Memory (write test). The server stores the column as Enum8('a' = 1, 'b' = 2).
  • Sample data for all these tables: one row ('b', 7)

Found by automated analysis of the client (Polyglot AI) while working on #3342. Verified against a live server, not only by reading the code.

Vorherrschende Sprache
Java
Sterne
1.6k
Forks
649
Ø Merge
1 T. 20 Std.
Gemergte PRs (30 T.)
39

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus ClickHouse/clickhouse-java

Alle Issues in ClickHouse/clickhouse-java

Ähnliche Issues

Weitere Issues zu Java

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.