[client-v2] ClickHouseColumn mis-parses a Tuple whose JSON element is not last: later elements are swallowed as JSON parameters
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in ClickHouseColumn.readColumn, at the 0.9.5 lines 508-516 or the corresponding 0.10.0 code, and inspect how JSON parameters are detected before the generic branch. Re-run the supplied ClickHouseColumn.of reproductions, especially Tuple(JSON, FixedString(3)) and Tuple(JSON, Decimal(10, 2)); done means later tuple elements remain separate and adjacent JSON parameters still parse correctly.
Written by the indexing model from the issue text.
Description
Summary
ClickHouseColumn.of(...) mis-parses a Tuple whose JSON element is not the last one: every element after the JSON up to the next closing parenthesis is swallowed as the JSON column's parameter list. Depending on what follows, the Tuple either silently loses elements or the parse throws Unknown data type.
Reproduced on clickhouse-data 0.9.5 and 0.10.0 (latest published), OpenJDK 17. No server needed.
Reproduce
import com.clickhouse.data.ClickHouseColumn;
ClickHouseColumn c = ClickHouseColumn.of("x", "Tuple(JSON, FixedString(3))");
c.getNestedColumns().size(); // 1 (expected 2)
c.getNestedColumns().get(0).getOriginalTypeName(); // "JSON, FixedString(3)"
ClickHouseColumn.of("x", "Tuple(JSON, Decimal(10, 2))");
// java.lang.IllegalArgumentException: Unknown data type: 2
Output of the snippet above (jshell, same on both versions):
Tuple(JSON, FixedString(3)) -> nested=1 first="JSON, FixedString(3)"
Tuple(JSON, Decimal(10, 2)) -> java.lang.IllegalArgumentException: Unknown data type: 2
Tuple(JSON, Int32) -> nested=2 (ok: nothing with parentheses after JSON)
Tuple(Int32, JSON) -> nested=2 (ok: JSON is last)
Tuple(FixedString(3), JSON) -> nested=2 (ok)
JSON(max_dynamic_paths=10) -> [max_dynamic_paths=10] (ok: the '(' is adjacent)
So the trigger is a non-final JSON element followed, anywhere later in the same Tuple, by a type that has parentheses.
Root cause
ClickHouseColumn.readColumn, 0.9.5 lines 508-516 (0.10.0: from line 553, unchanged):
} else if (args.startsWith(KEYWORD_JSON, i)) {
int index = args.indexOf('(', i + KEYWORD_JSON.length()); // scans to the END of the type string
if (index > i) {
i = ClickHouseUtils.skipBrackets(args, index, len, '(');
String originalTypeName = args.substring(startIndex, i);
...
parseJSONColumn(args.substring(index + 1, i - 1), nestedColumns, parameters);
indexOf('(') starts after the keyword but is not bounded to the next character, so inside a Tuple the ( of a later element is found. skipBrackets then consumes through that element's matching ), the span JSON, FixedString(3) becomes one JSON column, and 3 is fed to parseJSONColumn as its parameters. The enclosing Tuple parser resumes after the swallowed text, sees its own ), and ends with one element fewer.
- With
FixedString(3):parseJSONColumn("3")reads3as a path name, finds no type, and returns silently, so the Tuple is short with no error. - With
Decimal(10, 2):parseJSONColumn("10, 2")reads10as a path name and tries to parse2as its type:Unknown data type: 2.
Impact
Client.getTableSchema() fails for any table containing such a column (second case), or returns a Tuple with fewer nested columns than the type text declares (first case). A writer that serializes from the parsed columns but sends getOriginalTypeName() in a RowBinaryWithNamesAndTypes header then writes short rows against a header that promises more, which corrupts or fails the batch with no indication of the cause.
Suggested fix
Treat ( as the JSON parameter list only when it immediately follows the keyword, e.g. index == i + KEYWORD_JSON.length() (optionally after whitespace), matching the generic branch below it, which only calls readParameters when the current character is (.
- Dominant language
- Java
- Stars
- 1.6k
- Forks
- 637
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 28
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ClickHouse/clickhouse-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ClickHouse/clickhouse-java#3143 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
ClickHouse/clickhouse-java#3111 ·
-
bug client-api-v2 test
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
ClickHouse/clickhouse-java#3076 ·
-
area:sql-parser bug client-v1
Difficulty 1/5 1-3 hours Newbie friendliness 92/100
ClickHouse/clickhouse-java#3066 ·
-
area:general bug client-api-v2 jdbc jdbc-v2
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/clickhouse-java#3063 ·
All issues in ClickHouse/clickhouse-java
Similar issues
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 90/100
apache/cloudstack#14222 ·
-
[BUG]茶杯方块在取茶时会引发崩溃 Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
1.0.0-alpha2 Type/Improvement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
wso2/dpdp-accelerator#272 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
area/frontend
Difficulty 2/5 1-3 hours Newbie friendliness 65/100