Hacktoberfest 2026:维护者为十月标记出来的 issue,仍然开放、适合新手。 浏览 Hacktoberfest issue

Simplified filter expression has a Null type instead of Int64 type across the FFI layer

未关闭
#1,551 11 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
55/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
冷清
技术栈
python, rust

调研方向

先从链接的 MRE 开始,特别是 repro.py 和 repro_provider 目录,然后比较 main 和 datafusion52 分支。运行 maturin develop 和 python repro.py 步骤以复现 LIMIT 5 panic,并运行针对 SELECT * FROM dummy_table WHERE a < 5 的相应 cargo test,以比较有无 FFI 层时的行为。完成的标准是 FFI 查询能够成功执行且不出现 Null-versus-Int64 assertion。

由索引模型根据 Issue 内容生成。

描述

bug

Describe the bug

A custom table provider for a ParquetSource with a trivial catalog and an Int64 column yields some errors when a SQL query has a filter with a literal limit on that column of the form

assertion `left == right` failed: Simplified expression should have the same data type as the original
  left: Null
 right: Int64

The error does not occur when using datafusion-python 52; it also does not occur when running the query purely in a Rust SessionContext; the backtrace for the above error shows it coming from datafusion-ffi code as well.

This may of course not be a bug, but instead some bad practice that the version 52 set of crates tolerates but which is now invalid. A MRE of this custom table provider can be found in the public repo https://github.com/jwimberl/datafusion_python_53_int64filter_repro, which contains

  • a non-working datafusion53 version (in branch main)
  • a baseline working version (in branch datafusion52)

and a canned dummy dataset. The README.md of this repo has more details.

To Reproduce

In the main branch, build the py_repro_provider crate with maturin develop and run python repro.py. This loads the dummy dataset as a table dummy_table and runs two queries

  • SELECT * FROM dummy_table LIMIT 1, which is successful
  • SELECT * FROM dummy_table LIMIT 5, which panics

Itss output should be something like

Successful query:
   a   b
0  0  42
Unsuccesful query:

thread '<unnamed>' panicked at /home/jwimberley/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-expr-53.1.0/src/simplifier/mod.rs:76:17:
assertion `left == right` failed: Simplified expression should have the same data type as the original
  left: Null
 right: Int64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

followed by backtrace information.

Expected behavior

In the datafusion52 branch, build thepy_repro_provider with maturin develop and run python repro.py. It runs the same two queries, and its output should be

Successful query:
   a   b
0  0  42
Also successful query:
   a   b
0  0  42
1  1  42
2  2  42
3  3  42
4  4  42

Additional context

In either the main branch or datafusion52 branch, the Rust code for the table provider is in the directory repro_provider, and there is a corresponding cargo test that runs SELECT * FROM dummy_table WHERE a < 5. Without the FFI layer, this is successful with both datafusion 52 and 53.

主要语言
Python
星标
605
派生
176
平均合并
1 天 23 小时
30 天内合并 PR
8

贡献指南

这个仓库没有索引到贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

apache/datafusion-python 的其他 Issue

查看 apache/datafusion-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。