`inspect.files()` and the other file metadata tables raise `ArrowInvalid` when a snapshot has no manifests

未关闭 适合新手
  • 哪里不对？
#4,269 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@YuangGao 已经在做这个了。

开始于 2026年11月3日。

  • #4272 来自 @YuangGao —— 未关闭

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
72/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python, sqlite
领域
data

调研方向

从 pyiceberg/table/inspect.py 开始：_files 以 pa.concat_tables(results) 结尾，_all_files 也是如此，因此当清单列表为空时两者都会失败。运行复现脚本，它会向 SqlCatalog 表追加一个空的 Arrow 表，然后调用 inspect.files()。当 files()、data_files()、delete_files() 以及 all_* 系列变体返回带有 files 架构的空表，而不是抛出 ArrowInvalid 时即完成，包括在用空表执行 overwrite 之后。

由索引模型根据 Issue 内容生成。

描述

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Appending an empty Arrow table creates a snapshot whose manifest list has no manifests. After that, inspect.files(), data_files(), delete_files(), all_files(), all_data_files(), and all_delete_files() all raise ArrowInvalid: Must pass at least one table. I expected each of them to return an empty table with the files schema.

import tempfile

import pyarrow as pa
from pyiceberg.catalog.sql import SqlCatalog

warehouse = tempfile.mkdtemp()
catalog = SqlCatalog("t", uri=f"sqlite:///{warehouse}/catalog.db", warehouse=f"file://{warehouse}")
catalog.create_namespace("n")
df = pa.table({"x": pa.array([], pa.int64())})
table = catalog.create_table("n.t", schema=df.schema)
table.append(df)
table.inspect.files()  # ArrowInvalid: Must pass at least one table

Traceback (most recent call last):
  File "repro.py", line 12, in <module>
    table.inspect.files()  # ArrowInvalid: Must pass at least one table
    ^^^^^^^^^^^^^^^^^^^^^
  File "pyiceberg/table/inspect.py", line 900, in files
    return self._files(snapshot_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyiceberg/table/inspect.py", line 888, in _files
    return pa.concat_tables(results)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow/table.pxi", line 6344, in pyarrow.lib.concat_tables
  File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Must pass at least one table

overwrite() with an empty table after a non-empty append also leaves a snapshot with no manifests, and files() fails the same way.

_files reads one table per manifest and ends with pa.concat_tables(results), so an empty manifest list calls pa.concat_tables([]). _all_files does the same at L981. Both already return an empty table one level up: _files when the table has no current snapshot (L876-L877), and _all_files when the table has no snapshots (L968-L970). Could they return the same empty table when there are no manifests?

This reproduces on 0.12.0 and on main at e3564a54c10960307147377c578680276865ed18.

Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
主要语言
Python
星标
1.2k
派生
654
平均合并
2 天 11 小时
30 天内合并 PR
122

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 有 Pull Request 模板
  • 没有贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

apache/iceberg-python 的其他 Issue

查看 apache/iceberg-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。