[jdbc-v2] User-Agent wraps detected frameworks in double parentheses: ((apache.spark))
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 2/5
- Geschätzter Aufwand
- 1-3 Stunden
- Anfängerfreundlichkeit
- 70/100
Rechercherichtung
Die doppelte Gruppierung entsteht an zwei Stellen in jdbc-v2: Driver.FrameworksDetection.getFrameworksDetected() in jdbc-v2/src/main/java/com/clickhouse/jdbc/Driver.java umschließt jeden Namen in Klammern, und ConnectionImpl in jdbc-v2 umschließt das zusammengefügte Ergebnis erneut. Entferne die Klammerung pro Name nur in der jdbc-v2-Kopie und lasse DriverV1.FrameworksDetection in jdbc-v1 unverändert. Fertig ist es, wenn ein Stack mit org.apache.spark genau eine Gruppe wie (apache.spark) liefert und ein Stack ohne Framework weiterhin keine Gruppe liefert. Ein Test in einem Paket org.apache.spark muss zuerst das statische Feld frameworksDetected zurücksetzen.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Description
When jdbc-v2 detects a framework on the call stack (apache.spark, apache.flink, apache.nifi), it adds the framework names to the User-Agent header inside two levels of parentheses:
jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
With two frameworks on the stack, each name gets its own inner group:
jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.flink); (apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
Nested comments are valid HTTP (RFC 9110), so requests do not fail. But the header is read by tools that parse system.query_log.http_user_agent (see #2775), and a single comment group is the shape that the rest of the header uses.
Root cause
Two places add parentheses:
Driver.FrameworksDetection.getFrameworksDetected()wraps each name:inferredFrameworks.add(String.format("(%s)", framework)), then joins with"; ".
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/jdbc-v2/src/main/java/com/clickhouse/jdbc/Driver.java#L50-L55ConnectionImplwraps the result again:clientName += " (" + detectedFrameworks + ")".
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/jdbc-v2/src/main/java/com/clickhouse/jdbc/ConnectionImpl.java#L101-L107
FrameworksDetection is the same code as DriverV1.FrameworksDetection in jdbc-v1. jdbc-v1 appends the result directly to the product name, with no outer group, so there the inner parentheses are the only group:
https://github.com/ClickHouse/clickhouse-java/blob/5392b642e5ca0375a77b8cc0f15b29b97a8f772f/clickhouse-jdbc/src/main/java/com/clickhouse/jdbc/internal/ClickHouseConnectionImpl.java#L287-L291
jdbc-v2 kept the per-name parentheses and added an outer group, so the names are wrapped twice.
Suggested fix
In jdbc-v2 only, remove one level. For example, add the plain name in Driver.FrameworksDetection (inferredFrameworks.add(framework)) and keep the outer group in ConnectionImpl. Result:
- one framework:
jdbc-v2/<v> (apache.spark) clickhouse-java-v2/<v> ... - two frameworks:
jdbc-v2/<v> (apache.flink; apache.spark) clickhouse-java-v2/<v> ...
Cases that must not change:
- No framework on the stack: no group (today:
jdbc-v2/<v> clickhouse-java-v2/<v> ...). - jdbc-v1 (
DriverV1.FrameworksDetection): it has no outer group, so it needs its per-name parentheses.
A unit test can call Driver.FrameworksDetection.getFrameworksDetected() from a class in an org.apache.spark.* package. The result is cached in a static field, so the test must reset FrameworksDetection.frameworksDetected (package-private) first.
Related: #3213 (open) changes the client-v2 part of the header to the standard format for #3202. It does not change the jdbc-v2 framework group and says this is a separate issue. A maintainer could also decide to move the frameworks into the new client_metadata key/value pairs. That changes the jdbc-v2 output and needs a key name, so it is a design decision, not part of this bug.
Steps to reproduce
- Put a class in a package whose name contains
apache.spark(for exampleorg.apache.spark.repro) and open a jdbc-v2 connection from it. - Run a query that has a unique marker, then
SYSTEM FLUSH LOGS. - Read
http_user_agentfor that query fromsystem.query_log.
Error Log or Exception StackTrace
No exception. Observed http_user_agent values (one JVM per row, because detection result is cached per JVM):
[spark on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[spark, client_name=my-app/1.0] my-app/1.0 jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[spark -> flink on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT ((apache.flink); (apache.spark)) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
[no framework on stack] jdbc-v2/0.12.0-rc1-SNAPSHOT clickhouse-java-v2/0.12.0-rc1-SNAPSHOT (Linux; jvm:17.0.20.1) Apache-HttpClient/5.6.4
Expected Behaviour
One comment group with the framework names:
jdbc-v2/0.12.0-rc1-SNAPSHOT (apache.spark) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT ...
jdbc-v2/0.12.0-rc1-SNAPSHOT (apache.flink; apache.spark) clickhouse-java-v2/0.12.0-rc1-SNAPSHOT ...
Code Example
package org.apache.spark.repro; // "apache.spark" in the stack trace triggers detection
import java.sql.*;
import java.util.*;
public class SparkApp {
public static void main(String[] args) throws Exception {
Properties p = new Properties();
p.setProperty("user", "default");
p.setProperty("password", "<password>");
String marker = "ua_probe_" + UUID.randomUUID().toString().replace("-", "");
try (Connection c = new com.clickhouse.jdbc.Driver().connect("jdbc:clickhouse://localhost:8123/default", p);
Statement s = c.createStatement()) {
s.executeQuery("SELECT '" + marker + "'").close();
s.execute("SYSTEM FLUSH LOGS");
try (ResultSet rs = s.executeQuery("SELECT http_user_agent FROM system.query_log"
+ " WHERE type = 'QueryFinish' AND query LIKE '%" + marker + "%'"
+ " AND query NOT LIKE '%query_log%' LIMIT 1")) {
rs.next();
System.out.println(rs.getString(1));
// prints: jdbc-v2/<v> ((apache.spark)) clickhouse-java-v2/<v> (...)
}
}
}
}
For the two-framework case, call the same code from a class in org.apache.flink.repro that SparkApp calls.
Configuration
Client Configuration
// defaults; framework detection is enabled by default
Environment
- Cloud
- Client version: jdbc-v2
0.12.0-rc1-SNAPSHOT, built frommainat 5392b642e5ca0375a77b8cc0f15b29b97a8f772f - Language version: Java 17.0.20.1
- OS: Linux
ClickHouse Server
- ClickHouse Server version: 26.9.1.1629
- ClickHouse Server non-default settings, if any: none relevant
CREATE TABLEstatements for tables involved: none (SELECT '<marker>'only)
Found by automated analysis (Polyglot AI) while working on #3202. Verified by running the code above against a live ClickHouse server, not only by reading the code.
- Vorherrschende Sprache
- Java
- Sterne
- 1.6k
- Forks
- 640
- Ø Merge
- 1 T. 13 Std.
- Gemergte PRs (30 T.)
- 41
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Hat eine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus ClickHouse/clickhouse-java
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
ClickHouse/clickhouse-java#3212 ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug client-api-v2 test
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 92/100
ClickHouse/clickhouse-java#3076 ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:sql-parser bug client-v1
Schwierigkeit 1/5 1-3 Stunden Anfängerfreundlichkeit 92/100
ClickHouse/clickhouse-java#3066 ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug client-api-v2 jdbc-v2
Schwierigkeit 1/5 1-3 Stunden Anfängerfreundlichkeit 78/100
ClickHouse/clickhouse-java#2957 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug client-v1 wontfix
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
ClickHouse/clickhouse-java#2895 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in ClickHouse/clickhouse-java
Ähnliche Issues
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
scylladb/scylla-cdc-source-connector#350 ·
Maintainer antworten meist innerhalb von 1 Tag
-
agent:working documentation enhancement go:build lane:needs-decision
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
Sanne/incus-spawn#1435 · 3 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
area:shuffle bug priority:high requires-triage
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
apache/datafusion-comet#7062 ·
Maintainer antworten meist innerhalb von 1 Tag
-
wait for review
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 68/100
Maintainer antworten meist innerhalb von 1 Tag
-
enhancement good first issue up-for-grabs
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 80/100
AnikethTS/ChangeRadar#22 ·
Maintainer antworten meist innerhalb von 1 Tag