[client-v2] Enum type name without explicit values (Enum8('a', 'b')) fails to parse: Invalid enum entry

Ouverte
#3,344 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
Une demi-journée
Accessibilité débutants
55/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
java
Domaine
databases

Piste de recherche

Start in clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java, constructor at lines 37-53, which is called from ClickHouseColumn.update() at line 181. Read the existing negative cases in ClickHouseColumnTest.testEnum before changing the parsing. Done when ClickHouseColumn.of("e", "Enum8('a', 'b')") yields the same constants as Enum8('a' = 1, 'b' = 2), names containing '=' parse correctly, and unquoted or mixed lists are still rejected.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

area:data-type bug client-api-v2 jdbc-v2

Description

The server accepts Enum elements without a number and assigns the numbers itself. The client type-name parser rejects them. ClickHouseColumn.of("e", "Enum8('a', 'b')") throws IllegalArgumentException: Invalid enum entry: 'a'. The same occurs for Enum(...), Enum16(...), for a list with a start value (Enum8('a' = 5, 'b')), and when the type is inside Nullable, Array, Tuple or Map.

Server docs (Enum): "you could omit numbers. ClickHouse will assign consecutive numbers automatically. Numbers are assigned starting from 1 by default." and "You can also specify legal starting number for the first name."

An element name that contains = and has no number gives a different error: Enum8('x=y', 'b') throws NumberFormatException: For input string: "y'".

Where it is reachable: only type names that the application writes:

  • client-v2: ClickHouseColumn.of(...) in a TableSchema for Client.newBinaryFormatReader(response, schema) (RowBinary without a header), RowBinaryFormatWriter, or register(Class, schema).
  • jdbc-v2: Connection.createArrayOf(typeName, …) and Connection.createStruct("Tuple(…)", …).

Type names from the server (RowBinaryWithNamesAndTypes / Native header, getTableSchema) always have explicit numbers, so they are not affected. The failure is an exception, not wrong data.

Root cause

ClickHouseEnum(Collection<String>) (clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseEnum.java:37-53, called from ClickHouseColumn.update() at ClickHouseColumn.java:181) requires the form name=value for each element. It splits on p.lastIndexOf('=') and throws Invalid enum entry when there is no = (line 49). For a name that contains =, lastIndexOf finds the = inside the quoted name, and Integer.parseInt fails on the rest of the name (line 47).

Suggested fix

In ClickHouseEnum(Collection<String>), read each element as a quoted name (with '' and \' escapes), then an optional = <integer>. Do not use lastIndexOf('='). Assign the numbers as the server does (autoAssignNumberForEnum in src/DataTypes/DataTypeEnum.cpp):

  • an element without a number at position i gets base + i. base is 1, or the number of the first element if the first element has a number;
  • the server accepts a mixed list only if all elements after the first have no number, or all elements after the first have a number.

Behavior that must not change (the server rejects these too):

  • unquoted names, for example Enum8(aa, 1) (existing negative case in ClickHouseColumnTest.testEnum). Server: Code: 223 … Elements of Enum data type must be of form: 'name' = number or 'name';
  • other mixed lists, for example Enum8('a' = 5, 'b', 'c' = 10, 'd'). Server: Code: 223 … All elements of Enum data type must be of form ….

Related: #3342 (width of a bare Enum(...)). If the width of a bare Enum comes from the element numbers, assign the implicit numbers first: Enum('a' = 127, 'b') is Enum16('a' = 127, 'b' = 128) on the server.

Steps to reproduce
  1. Build main (f73e372bc).
  2. Call ClickHouseColumn.of("e", "Enum8('a', 'b')"), or use that type name in a TableSchema for newBinaryFormatReader / RowBinaryFormatWriter.
  3. The call throws IllegalArgumentException: Invalid enum entry: 'a'. With Enum8('a' = 1, 'b' = 2) the same code works.
Error Log or Exception StackTrace
java.lang.IllegalArgumentException: Invalid enum entry: 'a'
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:49)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)
	at com.clickhouse.data.ClickHouseColumn.readColumn(ClickHouseColumn.java:732)
	at com.clickhouse.data.ClickHouseColumn.of(ClickHouseColumn.java:763)

java.lang.NumberFormatException: For input string: "y'"     (type Enum8('x=y', 'b'))
	at java.base/java.lang.Integer.parseInt(Integer.java:668)
	at com.clickhouse.data.ClickHouseEnum.<init>(ClickHouseEnum.java:47)
	at com.clickhouse.data.ClickHouseColumn.update(ClickHouseColumn.java:180)

Server type (SELECT toTypeName(defaultValueOfTypeName('<type>'))) compared with the client parser (ClickHouseColumn.of("e", "<type>")):

type                                server                                 client
Enum8('a' = 1, 'b' = 2)             Enum8('a' = 1, 'b' = 2)                Enum8 {a=1, b=2}
Enum('a', 'b')                      Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum8('a', 'b')                     Enum8('a' = 1, 'b' = 2)                Invalid enum entry: 'a'
Enum16('a', 'b')                    Enum16('a' = 1, 'b' = 2)               Invalid enum entry: 'a'
Enum8('a' = 5, 'b')                 Enum8('a' = 5, 'b' = 6)                Invalid enum entry: 'b'
Enum8('a', 'b' = 5)                 Enum8('a' = 1, 'b' = 5)                Invalid enum entry: 'a'
Enum16('a' = 1000, 'b')             Enum16('a' = 1000, 'b' = 1001)         Invalid enum entry: 'b'
Enum8('a' = -3, 'b', 'c')           Enum8('a' = -3, 'b' = -2, 'c' = -1)    Invalid enum entry: 'b'
Enum8('Query''Start', 'b')          Enum8('Query\'Start' = 1, 'b' = 2)     Invalid enum entry: 'Query''Start'
Enum8('x=y', 'b')                   Enum8('x=y' = 1, 'b' = 2)              NumberFormatException: "y'"
Nullable(Enum8('a', 'b'))           Nullable(Enum8('a' = 1, 'b' = 2))      Invalid enum entry: 'a'
Array(Enum8('a', 'b'))              Array(Enum8('a' = 1, 'b' = 2))         Invalid enum entry: 'a'
Tuple(Enum8('a', 'b'), UInt8)       Tuple(Enum8('a' = 1, 'b' = 2), UInt8)  Invalid enum entry: 'a'
Map(String, Enum8('a', 'b'))        Map(String, Enum8('a' = 1, 'b' = 2))   Invalid enum entry: 'a'
Enum8('a' = 5, 'b', 'c' = 10, 'd')  error (Code: 223)                      Invalid enum entry: 'b'   (both reject: OK)
Enum8(aa, 1)                        error (Code: 223)                      Invalid enum entry: aa    (both reject: OK)

End-to-end results (same program):

client-v2 read, RowBinary, schema e Enum8('a' = 1, 'b' = 2)  -> e=b u=7
client-v2 read, RowBinary, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
client-v2 RowBinaryFormatWriter, schema e Enum8('a' = 1, 'b' = 2) -> inserted OK
client-v2 RowBinaryFormatWriter, schema e Enum8('a', 'b')          -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createArrayOf("Enum8('a', 'b')", …)                  -> IllegalArgumentException: Invalid enum entry: 'a'
jdbc-v2 createStruct("Tuple(Enum8('a', 'b'), UInt8)", …)     -> IllegalArgumentException: Invalid enum entry: 'a'
Expected Behaviour

The client parses the type name with the numbers that the server assigns. Enum8('a', 'b') gives the same constants as Enum8('a' = 1, 'b' = 2), and the reader and writer work with it.

Code Example
// Parser
ClickHouseColumn.of("e", "Enum8('a', 'b')"); // throws IllegalArgumentException: Invalid enum entry: 'a'

// client-v2 read with an application schema
TableSchema schema = new TableSchema(Arrays.asList(
        ClickHouseColumn.of("e", "Enum8('a', 'b')"),   // throws here
        ClickHouseColumn.of("u", "UInt8")));
try (QueryResponse resp = client.query(
            "SELECT CAST('b', 'Enum8(\\'a\\' = 1, \\'b\\' = 2)') AS e, toUInt8(7) AS u",
            new QuerySettings().setFormat(ClickHouseFormat.RowBinary)).get();
     ClickHouseBinaryFormatReader r = client.newBinaryFormatReader(resp, schema)) {
    r.next();
    System.out.println(r.getString("e"));
}
Configuration
Client Configuration
Client client = new Client.Builder()
        .addEndpoint("http://<host>:8123")
        .setUsername("default").setPassword("<password>")
        .build();
Environment
  • Cloud
  • Client version: main at f73e372bc (0.12.0-rc1-SNAPSHOT): clickhouse-data, client-v2, jdbc-v2
  • Language version: OpenJDK 17.0.21
  • OS: Linux (Docker)
ClickHouse Server
  • ClickHouse Server version: 26.9.1.1629
  • ClickHouse Server non-default settings, if any: none
  • CREATE TABLE statements for tables involved: CREATE TABLE enum_impl_w (e Enum8('a', 'b'), u UInt8) ENGINE = Memory (write test). The server stores the column as Enum8('a' = 1, 'b' = 2).
  • Sample data for all these tables: one row ('b', 7)

Found by automated analysis of the client (Polyglot AI) while working on #3342. Verified against a live server, not only by reading the code.

Langage dominant
Java
Étoiles
1.6k
Forks
649
Merge moyen
1 j 20 h
PR mergées (30 j)
39

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de ClickHouse/clickhouse-java

Toutes les issues de ClickHouse/clickhouse-java

Issues similaires

Plus d'issues Java

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.