`inspect.files()` and the other file metadata tables raise `ArrowInvalid` when a snapshot has no manifests

Open Beginner friendly
  • Is something wrong?
#4,269 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@YuangGao is already working on this.

Since Nov 3, 2026.

  • #4272 by @YuangGao — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, sqlite
Domain
data

Research direction

Start in pyiceberg/table/inspect.py: _files ends with pa.concat_tables(results), and _all_files does the same, so both fail when the manifest list is empty. Run the reproduction script, which appends an empty Arrow table to a SqlCatalog table and then calls inspect.files(). Done when files(), data_files(), delete_files() and the all_* variants return an empty table with the files schema instead of raising ArrowInvalid, including after an overwrite with an empty table.

Written by the indexing model from the issue text.

Description

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Appending an empty Arrow table creates a snapshot whose manifest list has no manifests. After that, inspect.files(), data_files(), delete_files(), all_files(), all_data_files(), and all_delete_files() all raise ArrowInvalid: Must pass at least one table. I expected each of them to return an empty table with the files schema.

import tempfile

import pyarrow as pa
from pyiceberg.catalog.sql import SqlCatalog

warehouse = tempfile.mkdtemp()
catalog = SqlCatalog("t", uri=f"sqlite:///{warehouse}/catalog.db", warehouse=f"file://{warehouse}")
catalog.create_namespace("n")
df = pa.table({"x": pa.array([], pa.int64())})
table = catalog.create_table("n.t", schema=df.schema)
table.append(df)
table.inspect.files()  # ArrowInvalid: Must pass at least one table

Traceback (most recent call last):
  File "repro.py", line 12, in <module>
    table.inspect.files()  # ArrowInvalid: Must pass at least one table
    ^^^^^^^^^^^^^^^^^^^^^
  File "pyiceberg/table/inspect.py", line 900, in files
    return self._files(snapshot_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyiceberg/table/inspect.py", line 888, in _files
    return pa.concat_tables(results)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow/table.pxi", line 6344, in pyarrow.lib.concat_tables
  File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Must pass at least one table

overwrite() with an empty table after a non-empty append also leaves a snapshot with no manifests, and files() fails the same way.

_files reads one table per manifest and ends with pa.concat_tables(results), so an empty manifest list calls pa.concat_tables([]). _all_files does the same at L981. Both already return an empty table one level up: _files when the table has no current snapshot (L876-L877), and _all_files when the table has no snapshots (L968-L970). Could they return the same empty table when there are no manifests?

This reproduces on 0.12.0 and on main at e3564a54c10960307147377c578680276865ed18.

Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Dominant language
Python
Stars
1.2k
Forks
664
Avg merge
2d 10h
Merged PRs (30d)
131

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.