[client-v2] Enum type name without explicit values (Enum8('a', 'b')) fails to parse: Invalid enum entry

Aperta
#3,344 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
Mezza giornata
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
java
Ambito
databases

Direzione di ricerca

Start in clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java, constructor at lines 37-53, which is called from ClickHouseColumn.update() at line 181. Read the existing negative cases in ClickHouseColumnTest.testEnum before changing the parsing. Done when ClickHouseColumn.of("e", "Enum8('a', 'b')") yields the same constants as Enum8('a' = 1, 'b' = 2), names containing '=' parse correctly, and unquoted or mixed lists are still rejected.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area:data-type bug client-api-v2 jdbc-v2

Description

The server accepts Enum elements without a number and assigns the numbers itself. The client type-name parser rejects them. ClickHouseColumn.of("e", "Enum8('a', 'b')") throws IllegalArgumentException: Invalid enum entry: 'a'. The same occurs for Enum(...), Enum16(...), for a list with a start value (Enum8('a' = 5, 'b')), and when the type is inside Nullable, Array, Tuple or Map.

Server docs (Enum): "you could omit numbers. ClickHouse will assign consecutive numbers automatically. Numbers are assigned starting from 1 by default." and "You can also specify legal starting number for the first name."

An element name that contains = and has no number gives a different error: Enum8('x=y', 'b') throws NumberFormatException: For input string: "y'".

Where it is reachable: only type names that the application writes:

  • client-v2: ClickHouseColumn.of(...) in a TableSchema for Client.newBinaryFormatReader(response, schema) (RowBinary without a header), RowBinaryFormatWriter, or register(Class, schema).
  • jdbc-v2: Connection.createArrayOf(typeName, …) and Connection.createStruct("Tuple(…)", …).

Type names from the server (RowBinaryWithNamesAndTypes / Native header, getTableSchema) always have explicit numbers, so they are not affected. The failure is an exception, not wrong data.

Root cause

ClickHouseEnum(Collection<String>) (clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java:37-53, called from ClickHouseColumn.update() at ClickHouseColumn.java:181) requires the form name=value for each element. It splits on p.lastIndexOf('=') and throws Invalid enum entry when there is no = (line 49). For a name that contains =, lastIndexOf finds the = inside the quoted name, and Integer.parseInt fails on the rest of the name (line 47).

Suggested fix

In ClickHouseEnum(Collection<String>), read each element as a quoted name (with '' and \' escapes), then an optional = <integer>. Do not use lastIndexOf('='). Assign the numbers as the server does (autoAssignNumberForEnum in src/DataTypes/DataTypeEnum.cpp):

  • an element without a number at position i gets base + i. base is 1, or the number of the first element if the first element has a number;
  • the server accepts a mixed list only if all elements after the first have no number, or all elements after the first have a number.

Behavior that must not change (the server rejects these too):

  • unquoted names, for example Enum8(aa, 1) (existing negative case in ClickHouseColumnTest.testEnum). Server: Code: 223 … Elements of Enum data type must be of form: 'name' = number or 'name';
  • other mixed lists, for example Enum8('a' = 5, 'b', 'c' = 10, 'd'). Server: Code: 223 … All elements of Enum data type must be of form ….

Related: #3342 (width of a bare Enum(...)). If the width of a bare Enum comes from the element numbers, assign the implicit numbers first: Enum('a' = 127, 'b') is Enum16('a' = 127, 'b' = 128) on the server.

Steps to reproduce
  1. Build main (f73e372bc).
  2. Call ClickHouseColumn.of("e", "Enum8('a', 'b')"), or use that type name in a TableSchema for newBinaryFormatReader / RowBinaryFormatWriter.
  3. The call throws IllegalArgumentException: Invalid enum entry: 'a'. With Enum8('a' = 1, 'b' = 2) the same code works.
Error Log or Exception StackTrace
java.lang.IllegalArgumentException: Invalid enum entry: 'a'
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:49)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)
	at com.clickhouse.data.ClickHouseColumn.readColumn(ClickHouseColumn.java:732)
	at com.clickhouse.data.ClickHouseColumn.of(ClickHouseColumn.java:763)

java.lang.NumberFormatException: For input string: "y'"     (type Enum8('x=y', 'b'))
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:47)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)

Server type (SELECT toTypeName(defaultValueOfTypeName('<type>'))) compared with the client parser (ClickHouseColumn.of("e", "<type>")):

type                                server                                 client
Enum8('a' = 1, 'b' = 2)             Enum8('a' = 1, 'b' = 2)                Enum8 {a=1, b=2}
Enum('a', 'b')                      Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum8('a', 'b')                     Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum16('a', 'b')                    Enum16('a' = 1, 'b' = 2)               Invalid enum entry: 'a'
Enum8('a' = 5, 'b')                 Enum8('a' = 5, 'b' = 6)                Invalid enum entry: 'b'
Enum8('a', 'b' = 5)                 Enum8('a' = 1, 'b' = 5)                Invalid enum entry: 'a'
Enum16('a' = 1000, 'b')             Enum16('a' = 1000, 'b' = 1001)         Invalid enum entry: 'b'
Enum8('a' = -3, 'b', 'c')           Enum8('a' = -3, 'b' = -2, 'c' = -1)    Invalid enum entry: 'b'
Enum8('Query''Start', 'b')          Enum8('Query\'Start' = 1, 'b' = 2)     Invalid enum entry: 'Query''Start'
Enum8('x=y', 'b')                   Enum8('x=y' = 1, 'b' = 2)              NumberFormatException: "y'"
Nullable(Enum8('a', 'b'))           Nullable(Enum8('a' = 1, 'b' = 2))      Invalid enum entry: 'a'
Array(Enum8('a', 'b'))              Array(Enum8('a' = 1, 'b' = 2))         Invalid enum entry: 'a'
Tuple(Enum8('a', 'b'), UInt8)       Tuple(Enum8('a' = 1, 'b' = 2), UInt8)  Invalid enum entry: 'a'
Map(String, Enum8('a', 'b'))        Map(String, Enum8('a' = 1, 'b' = 2))   Invalid enum entry: 'a'
Enum8('a' = 5, 'b', 'c' = 10, 'd')  error (Code: 223)                      Invalid enum entry: 'b'   (both reject: OK)
Enum8(aa, 1)                        error (Code: 223)                      Invalid enum entry: aa    (both reject: OK)

End-to-end results (same program):

client-v2 read, RowBinary, schema e Enum8('a' = 1, 'b' = 2)  -> e=b u=7
client-v2 read, RowBinary, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
client-v2 RowBinaryFormatWriter, schema e Enum8('a' = 1, 'b' = 2) -> inserted OK
client-v2 RowBinaryFormatWriter, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createArrayOf("Enum8('a', 'b')", …)                  -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createStruct("Tuple(Enum8('a', 'b'), UInt8)", …)     -> IllegalArgumentException: Invalid enum entry: 'a'
Expected Behaviour

The client parses the type name with the numbers that the server assigns. Enum8('a', 'b') gives the same constants as Enum8('a' = 1, 'b' = 2), and the reader and writer work with it.

Code Example
// Parser
ClickHouseColumn.of("e", "Enum8('a', 'b')"); // throws IllegalArgumentException: Invalid enum entry: 'a'

// client-v2 read with an application schema
TableSchema schema = new TableSchema(Arrays.asList(
        ClickHouseColumn.of("e", "Enum8('a', 'b')"),   // throws here
        ClickHouseColumn.of("u", "UInt8")));
try (QueryResponse resp = client.query(
            "SELECT CAST('b', 'Enum8(\\'a\\' = 1, \\'b\\' = 2)') AS e, toUInt8(7) AS u",
            new QuerySettings().setFormat(ClickHouseFormat.RowBinary)).get();
     ClickHouseBinaryFormatReader r = client.newBinaryFormatReader(resp, schema)) {
    r.next();
    System.out.println(r.getString("e"));
}
Configuration
Client Configuration
Client client = new Client.Builder()
        .addEndpoint("http://<host>:8123")
        .setUsername("default").setPassword("<password>")
        .build();
Environment
  • Cloud
  • Client version: main at f73e372bc (0.12.0-rc1-SNAPSHOT): clickhouse-data, client-v2, jdbc-v2
  • Language version: OpenJDK 17.0.21
  • OS: Linux (Docker)
ClickHouse Server
  • ClickHouse Server version: 26.9.1.1629
  • ClickHouse Server non-default settings, if any: none
  • CREATE TABLE statements for tables involved: CREATE TABLE enum_impl_w (e Enum8('a', 'b'), u UInt8) ENGINE = Memory (write test). The server stores the column as Enum8('a' = 1, 'b' = 2).
  • Sample data for all these tables: one row ('b', 7)

Found by automated analysis of the client (Polyglot AI) while working on #3342. Verified against a live server, not only by reading the code.

Lingua principale
Java
Stelle
1.6k
Fork
649
Merge medio
1g 20h
PR unite (30g)
39

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ClickHouse/clickhouse-java

Tutte le issue di ClickHouse/clickhouse-java

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.