Implement Spark-compatible `xpath_boolean`

未关闭
  • 哪里不对？
#282 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
55/100
Issue 类型
功能
描述清晰度
描述清楚
活跃度
活跃
技术栈
rust, sql
领域
databases

调研方向

Start with Spark's XPathBoolean in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala to see the boolean conversion, then confirm the shared evaluator from the xpath issue exists before writing anything. The new function belongs in crates/spark/src/function/xml/, and crates/sqllogictest/test_files/xml/xpath_boolean.slt is the commented-out stub to enable. Done means that slt file passes the examples and edge cases above (node-set truthiness, number/string comparisons, NULL and non-foldable path errors), with #[user_doc] on the function. Note the prerequisite evaluator is still an open issue, so check it is landed first.

由索引模型根据 Issue 内容生成。

描述

enhancement

Part of #271 and apache/datafusion-comet#7300.

What the function does

xpath_boolean(xml, path) evaluates an XPath 1.0 expression and converts the result to BOOLEAN using the
XPath boolean() rules (XPathConstants.BOOLEAN). Both arguments are strings (collation-aware from 4.0), and
path must be foldable. The result is always nullable. Available since Spark 2.0.0. The expression is a
Predicate, so it can be used directly in WHERE.

Parsing, DTD and entity handling, namespaces, NULL and empty-string handling, invalid-XML and invalid-path
errors, and lazy path compilation are shared with the rest of the family. They are specified in the xpath
issue, and the implementation should reuse that evaluator. This issue covers only the boolean conversion.

Examples
SELECT xpath_boolean('<a><b>1</b></a>','a/b');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/c');
-- false
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "b"');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "c"');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b < 10');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b = 10');
-- true

These come from the ExpressionDescription, XPathExpressionSuite and the Hive udf_xpath_boolean golden
files run by HiveCompatibilitySuite. In UDFXPathUtilSuite, a/b[1]/text() is true and a/b[4] (no match)
is false.

Edge cases to test
  • Conversion rules from XPath 1.0 boolean():
    • a node-set is true if it is non-empty, whatever the text. <b>false</b> with a/b/text() gives true;
    • a number is false for 0, -0 and NaN, and true otherwise;
    • a string is true if it is non-empty, so string(a/c) on a missing node gives false;
    • a boolean result is used as is.
  • Comparisons between a node-set and a value are existential: a/b = 2 is true if any b equals 2. Also test
    number versus string comparison (a/b = "10" versus a/b = 10 with <b>10.0</b>).
  • NULL or '' for either argument gives NULL. A non-literal path fails with
    DATATYPE_MISMATCH.NON_FOLDABLE_INPUT.
  • Malformed XML and an invalid path raise the runtime errors described in the xpath issue. There is no
    false fallback.
  • Use in a filter (WHERE xpath_boolean(xml, 'a/b')) with NULL rows. NULL rows are filtered out.
Differences across Spark versions

None found beyond the shared ones listed in the xpath issue (4.0 moved to RuntimeReplaceable and added
collation support).

Implementation notes
  • Spark: XPathBoolean in sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala.
    It calls XPathBooleanEvaluator, which calls UDFXPathUtil.evalBoolean.
  • DataFusion core has no equivalent.
  • Put it in crates/spark/src/function/xml/ on top of the shared evaluator from the xpath issue.
    The existing crates/sqllogictest/test_files/xml/xpath_boolean.slt is a commented-out stub that should be
    enabled and extended. Document the function with #[user_doc].

This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.

主要语言
Rust
星标
7
派生
15
平均合并
16 小时 43 分钟
30 天内合并 PR
26

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

apache/datafusion-spark 的其他 Issue

查看 apache/datafusion-spark 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。