[jdbc-v2] User-Agent wraps detected frameworks in double parentheses: ((apache.spark))
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 70/100
Research direction
The double group comes from two places in jdbc-v2: Driver.FrameworksDetection.getFrameworksDetected() in jdbc-v2/src/main/java/com/clickhouse/jdbc/Driver.java wraps each name in parentheses, and ConnectionImpl in jdbc-v2 wraps the joined result again. Remove the per-name wrapping in the jdbc-v2 copy only, leaving DriverV1.FrameworksDetection in jdbc-v1 unchanged. Done when a stack containing org.apache.spark yields one group like (apache.spark), and a stack with no framework still yields no group. A test in an org.apache.spark package must reset the static frameworksDetected field first.
Written by the indexing model from the issue text.
Description
Description
When jdbc-v2 detects a framework on the call stack (apache.spark, apache.flink, apache.nifi), it adds the framework names to the User-Agent header inside two levels of parentheses:
jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
With two frameworks on the stack, each name gets its own inner group:
jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.flink); (apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
Nested comments are valid HTTP (RFC 9110), so requests do not fail. But the header is read by tools that parse system.query_log.http_user_agent (see #2775), and a single comment group is the shape that the rest of the header uses.
Root cause
Two places add parentheses:
Driver.FrameworksDetection.getFrameworksDetected()wraps each name:inferredFrameworks.add(String.format("(%s)", framework)), then joins with"; ".
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/jdbc-v2/src/main/java/com/clickhouse/jdbc/Driver.java#L50-L55ConnectionImplwraps the result again:clientName += " (" + detectedFrameworks + ")".
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/jdbc-v2/src/main/java/com/clickhouse/jdbc/ConnectionImpl.java#L101-L107
FrameworksDetection is the same code as DriverV1.FrameworksDetection in jdbc-v1. jdbc-v1 appends the result directly to the product name, with no outer group, so there the inner parentheses are the only group:
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/clickhouse-jdbc/src/main/java/com/clickhouse/jdbc/internal/ClickHouseConnectionImpl.java#L287-L291
jdbc-v2 kept the per-name parentheses and added an outer group, so the names are wrapped twice.
Suggested fix
In jdbc-v2 only, remove one level. For example, add the plain name in Driver.FrameworksDetection (inferredFrameworks.add(framework)) and keep the outer group in ConnectionImpl. Result:
- one framework:
jdbc-v2/<v> (apache.spark) clickhouse-java-v2/<v> ... - two frameworks:
jdbc-v2/<v> (apache.flink; apache.spark) clickhouse-java-v2/<v> ...
Cases that must not change:
- No framework on the stack: no group (today:
jdbc-v2/<v> clickhouse-java-v2/<v> ...). - jdbc-v1 (
DriverV1.FrameworksDetection): it has no outer group, so it needs its per-name parentheses.
A unit test can call Driver.FrameworksDetection.getFrameworksDetected() from a class in an org.apache.spark.* package. The result is cached in a static field, so the test must reset FrameworksDetection.frameworksDetected (package-private) first.
Related: #3213 (open) changes the client-v2 part of the header to the standard format for #3202. It does not change the jdbc-v2 framework group and says this is a separate issue. A maintainer could also decide to move the frameworks into the new client_metadata key/value pairs. That changes the jdbc-v2 output and needs a key name, so it is a design decision, not part of this bug.
Steps to reproduce
- Put a class in a package whose name contains
apache.spark(for exampleorg.apache.spark.repro) and open a jdbc-v2 connection from it. - Run a query that has a unique marker, then
SYSTEM FLUSH LOGS. - Read
http_user_agentfor that query fromsystem.query_log.
Error Log or Exception StackTrace
No exception. Observed http_user_agent values (one JVM per row, because detection result is cached per JVM):
[spark on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[spark, client_name=my-app/1.0] my-app/1.0 jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[spark -> flink on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.flink); (apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[no framework on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
Expected Behaviour
One comment group with the framework names:
jdbc-v2/0.12.0-rc1-SNAPSHOT (apache.spark) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT ...
jdbc-v2/0.12.0-rc1-SNAPSHOT (apache.flink; apache.spark) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT ...
Code Example
package org.apache.spark.repro; // "apache.spark" in the stack trace triggers detection
import java.sql.*;
import java.util.*;
public class SparkApp {
public static void main(String[] args) throws Exception {
Properties p = new Properties();
p.setProperty("user", "default");
p.setProperty("password", "<password>");
String marker = "ua_probe_" + UUID.randomUUID().toString().replace("-", "");
try (Connection c = new com.clickhouse.jdbc.Driver().connect("jdbc:clickhouse://localhost:8123/default", p);
Statement s = c.createStatement()) {
s.executeQuery("SELECT '" + marker + "'").close();
s.execute("SYSTEM FLUSH LOGS");
try (ResultSet rs = s.executeQuery("SELECT http_user_agent FROM system.query_log"
+ " WHERE type = 'QueryFinish' AND query LIKE '%" + marker + "%'"
+ " AND query NOT LIKE '%query_log%' LIMIT 1")) {
rs.next();
System.out.println(rs.getString(1));
// prints: jdbc-v2/<v> ((apache.spark)) clickhouse-java-v2/<v> (...)
}
}
}
}
For the two-framework case, call the same code from a class in org.apache.flink.repro that SparkApp calls.
Configuration
Client Configuration
// defaults; framework detection is enabled by default
Environment
- Cloud
- Client version: jdbc-v2
0.12.0-rc1-SNAPSHOT, built frommainat 5392b642e5ca0375a77b8cc0f15b29b97a8f772f - Language version: Java 17.0.20.1
- OS: Linux
ClickHouse Server
- ClickHouse Server version: 26.9.1.1629
- ClickHouse Server non-default settings, if any: none relevant
CREATE TABLEstatements for tables involved: none (SELECT '<marker>'only)
Found by automated analysis (Polyglot AI) while working on #3202. Verified by running the code above against a live ClickHouse server, not only by reading the code.
- Dominant language
- Java
- Stars
- 1.6k
- Forks
- 640
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 42
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ClickHouse/clickhouse-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ClickHouse/clickhouse-java#3212 ·
Maintainers usually reply within 1 day
-
bug client-api-v2 test
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
ClickHouse/clickhouse-java#3076 ·
Maintainers usually reply within 1 day
-
area:sql-parser bug client-v1
Difficulty 1/5 1-3 hours Newbie friendliness 92/100
ClickHouse/clickhouse-java#3066 ·
Maintainers usually reply within 1 day
-
bug client-api-v2 jdbc-v2
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
ClickHouse/clickhouse-java#2957 · 1 comment ·
Maintainers usually reply within 1 day
-
bug client-v1 wontfix
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ClickHouse/clickhouse-java#2895 · 1 comment ·
Maintainers usually reply within 1 day
All issues in ClickHouse/clickhouse-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/rocketmq-dashboard#6872 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ADORSYS-GIS/token-status-link#168 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ADORSYS-GIS/keycloak-oid4vp-plugin#232 ·
Maintainers usually reply within 2 days
-
Type/Improvement
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
wso2/product-integrator-mi#5099 ·
Maintainers usually reply within 1 day