Implement Spark-compatible `xpath_boolean`

Abierto
  • ¿Algo no está bien?
#282 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
55/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
rust, sql
Área
databases

Línea de trabajo

Start with Spark's XPathBoolean in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala to see the boolean conversion, then confirm the shared evaluator from the xpath issue exists before writing anything. The new function belongs in crates/spark/src/function/xml/, and crates/sqllogictest/test_files/xml/xpath_boolean.slt is the commented-out stub to enable. Done means that slt file passes the examples and edge cases above (node-set truthiness, number/string comparisons, NULL and non-foldable path errors), with #[user_doc] on the function. Note the prerequisite evaluator is still an open issue, so check it is landed first.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Part of #271 and apache/datafusion-comet#7300.

What the function does

xpath_boolean(xml, path) evaluates an XPath 1.0 expression and converts the result to BOOLEAN using the
XPath boolean() rules (XPathConstants.BOOLEAN). Both arguments are strings (collation-aware from 4.0), and
path must be foldable. The result is always nullable. Available since Spark 2.0.0. The expression is a
Predicate, so it can be used directly in WHERE.

Parsing, DTD and entity handling, namespaces, NULL and empty-string handling, invalid-XML and invalid-path
errors, and lazy path compilation are shared with the rest of the family. They are specified in the xpath
issue, and the implementation should reuse that evaluator. This issue covers only the boolean conversion.

Examples
SELECT xpath_boolean('<a><b>1</b></a>','a/b');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/c');
-- false
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "b"');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "c"');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b < 10');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b = 10');
-- true

These come from the ExpressionDescription, XPathExpressionSuite and the Hive udf_xpath_boolean golden
files run by HiveCompatibilitySuite. In UDFXPathUtilSuite, a/b[1]/text() is true and a/b[4] (no match)
is false.

Edge cases to test
  • Conversion rules from XPath 1.0 boolean():
    • a node-set is true if it is non-empty, whatever the text. <b>false</b> with a/b/text() gives true;
    • a number is false for 0, -0 and NaN, and true otherwise;
    • a string is true if it is non-empty, so string(a/c) on a missing node gives false;
    • a boolean result is used as is.
  • Comparisons between a node-set and a value are existential: a/b = 2 is true if any b equals 2. Also test
    number versus string comparison (a/b = "10" versus a/b = 10 with <b>10.0</b>).
  • NULL or '' for either argument gives NULL. A non-literal path fails with
    DATATYPE_MISMATCH.NON_FOLDABLE_INPUT.
  • Malformed XML and an invalid path raise the runtime errors described in the xpath issue. There is no
    false fallback.
  • Use in a filter (WHERE xpath_boolean(xml, 'a/b')) with NULL rows. NULL rows are filtered out.
Differences across Spark versions

None found beyond the shared ones listed in the xpath issue (4.0 moved to RuntimeReplaceable and added
collation support).

Implementation notes
  • Spark: XPathBoolean in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala.
    It calls XPathBooleanEvaluator, which calls UDFXPathUtil.evalBoolean.
  • DataFusion core has no equivalent.
  • Put it in crates/spark/src/function/xml/ on top of the shared evaluator from the xpath issue.
    The existing crates/sqllogictest/test_files/xml/xpath_boolean.slt is a commented-out stub that should be
    enabled and extended. Document the function with #[user_doc].

This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.

Lenguaje dominante
Rust
Estrellas
7
Forks
15
Merge medio
16 h 43 min
PR fusionados (30 d)
26

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/datafusion-spark

Todos los issues de apache/datafusion-spark

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.