[Java][FlightSQL] JDBC Driver fails when Looker formats the URL with ':direct=' instead of '://'
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 2/5
- Tiempo estimado
- 1-3 horas
- Aptitud para principiantes
- 38/100
Línea de trabajo
Comienza en flight/flight-sql/flight-sql-jdbc-driver, en ArrowFlightJdbcDriver.java, y después lee ArrowFlightJdbcDriverTest y sus pruebas de análisis de URL. Reproduce el caso legacy :direct= y verifica que acceptsURL, getUrlsArgs y connect gestionen el host, el puerto y los parámetros, mientras la suite existente sigue pasando.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Describe the enhancement requested
I am running a customer-hosted Looker instance and trying to connect to Dremio using the
Apache Arrow Flight SQL JDBC driver (flight-sql-jdbc-driver), registered in Looker via a
custom_jdbc_config.yml entry so the native Dremio dialect can be kept:
- name: dremio
dir_name: dremio
module_path: org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver
override_jdbc_url_subprotocol: arrow-flight-sql
override_jdbc_url_subprotocol only swaps the subprotocol token Looker plugs into its
existing, dialect-specific URL template for Dremio — it does not rewrite the rest of the
template. Looker's native Dremio dialect builds connection strings as:
jdbc:dremio:direct=<host>:<port>
so after the subprotocol override, Looker generates:
jdbc:arrow-flight-sql:direct=<host>:<port>
instead of the driver's expected format:
jdbc:arrow-flight-sql://<host>:<port>
Because ArrowFlightJdbcDriver parses the URL with java.net.URI.create(...) and strictly
expects the :// delimiter to separate scheme from host, it cannot extract a host from the
:direct= form, and the connection fails.
I tried a workaround by switching Looker's dialect dropdown to "Trino" (which emits the
standard :// format). While queries did run correctly over Arrow Flight, it broke the
Looker SQL Runner UI, since Looker then sends Trino-specific metadata queries to Dremio.
To use this driver properly with Looker, the native Dremio dialect needs to stay selected,
which means the driver needs to tolerate Looker's :direct= URL format.
Steps to Reproduce / Current Behavior
Passing a connection string containing :direct= to the driver:
jdbc:arrow-flight-sql:direct=<dremio-host>:<port>
fails instantly with:
Java::JavaSql::SQLException: URL must have a host. Expected format: jdbc:arrow-flight-sql://[host][:port][?param1=value&...]
Fix
Implemented and verified locally against flight/flight-sql-jdbc-core. Added a normalization
step to ArrowFlightJdbcDriver.java that rewrites the legacy :direct= delimiter to ://
before any URL parsing occurs, applied consistently at every entry point that parses the
connection string (acceptsURL, connect, getUrlsArgs):
// Some BI tools (e.g. Looker) generate connection strings using the legacy
// "jdbc:arrow-flight-sql:direct=<host>:<port>" syntax instead of the standard "://" delimiter.
// Normalize it so the URL can still be parsed as a valid URI.
private static final String LEGACY_DIRECT_DELIMITER = ":direct=";
private static String normalizeUrl(final String url) {
if (url != null && url.contains(LEGACY_DIRECT_DELIMITER)) {
return url.replace(LEGACY_DIRECT_DELIMITER, "://");
}
return url;
}
acceptsURL(String url)now normalizes before checking thejdbc:arrow-flight-sql:///
jdbc:arrow-flight://prefixes.connect(String url, Properties info)normalizes once up front and uses the normalized URL
for both argument parsing (getUrlsArgs) andArrowFlightConnection.createNewConnection.getUrlsArgs(String url)normalizes defensively as well, since it is@VisibleForTesting
and may be called directly.
This is a purely additive, backward-compatible change — URLs that don't contain :direct=
are returned unmodified.
Testing / Verification
Added two regression tests to ArrowFlightJdbcDriverTest:
testDriverUrlParsingMechanismShouldAcceptLegacyDirectDelimiter— verifiesacceptsURLand
getUrlsArgscorrectly parse host/port/params from ajdbc:arrow-flight-sql:direct=...URL.testShouldConnectWhenProvidedWithLegacyDirectDelimiterUrl— verifies a live connection can
be established end-to-end using the:direct=URL format against the test Flight SQL server.
Full ArrowFlightJdbcDriverTest suite: 18/18 tests passing, 0 failures, 0 errors.
Also validated end-to-end against a real customer-hosted Looker instance connecting to Dremio
with the native Dremio dialect (config above) — connection succeeded after applying the fix.
Additional deployment note (not part of this fix, but relevant to anyone testing it with Looker)
The flight-sql-jdbc-driver shaded jar is built with Java 17 bytecode by default
(maven.compiler.release=17 in the root POM), but Looker's bundled JRE is Java 11. Loading
the default build in Looker fails with:
UnsupportedClassVersionError: ... class file version 61.0, this version of the Java Runtime
only recognizes class file versions up to 55.0
To build a Looker-compatible jar, rebuild the driver and its Arrow-owned dependency closure
(arrow-format, arrow-memory-core, arrow-memory-netty(-buffer-patch), arrow-vector,
flight-core, flight-sql, flight-sql-jdbc-core, flight-sql-jdbc-driver) with:
mvn -pl format,memory/memory-core,memory/memory-netty,memory/memory-netty-buffer-patch,vector,flight/flight-core,flight/flight-sql,flight/flight-sql-jdbc-core,flight/flight-sql-jdbc-driver \
-DskipTests -Dmaven.compiler.release=11 -Dmaven.compiler.source=11 -Dmaven.compiler.target=11 \
clean install
(clean is required — Maven's incremental compiler otherwise skips recompilation when only
compiler flags change.) This may be worth calling out in the driver's documentation for users
deploying it to JVM-hosted BI tools that ship an older bundled runtime.
- Lenguaje dominante
- Java
- Estrellas
- 95
- Forks
- 154
- Merge medio
- 2 d 10 h
- PR fusionados (30 d)
- 11
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de apache/arrow-java
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
apache/arrow-java#1261 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
apache/arrow-java#1236 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
apache/arrow-java#1230 ·
-
Type: bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
apache/arrow-java#1205 ·
-
Type: bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
apache/arrow-java#1196 · 1 comentario ·
Todos los issues de apache/arrow-java
Issues similares
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
-
Two open-case totals on one screen: the Programs tile says 15,858 and the nav badge says 15,868 Abiertobug frontend maui-pilot
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
objectionary/eo-graphs#74 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100