Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Native Iceberg write silently falls back on Iceberg 1.12: ManifestFiles.read(ManifestFile, FileIO) was removed

Open Beginner friendly
  • Is something wrong?
#7,240 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
scala
Domain
backend, databases

Research direction

Start in IcebergReflection.scala at probeExecutorReflection and readDataFilesFromManifest, the two places named in the issue. Check the Iceberg 1.12 ManifestFiles.read signature and how the table's partition specs are available; then run the Iceberg write path with the 1.12 runtime. Done means native writes no longer fall back and the executor reflection probe resolves successfully.

Written by the indexing model from the issue text.

Description

requires-triage
Describe the bug

With iceberg-spark-runtime 1.12.0 every native Iceberg write falls back, logging only:

Comet cannot accelerate IcebergWriteExec because: executor-side Iceberg reflection did not resolve:
NoSuchMethodException: org.apache.iceberg.ManifestFiles.read(org.apache.iceberg.ManifestFile,org.apache.iceberg.io.FileIO)

Iceberg 1.12.0 removed the public 2-arg overload (javap on iceberg-spark-runtime-3.5_2.12):

1.11.0: public static ManifestReader<DataFile> read(ManifestFile, FileIO)
        public static ManifestReader<DataFile> read(ManifestFile, FileIO, Map<Integer, PartitionSpec>)
1.12.0: public static ManifestReader<DataFile> read(ManifestFile, FileIO, Map<Integer, PartitionSpec>)

Comet resolves the 2-arg form in two places in IcebergReflection.scala: the executor probe
(probeExecutorReflection, line 1836 on main) and readDataFilesFromManifest (line 2004). The
native Iceberg scan still works on 1.12.

Steps to reproduce

Any rewrite_data_files or INSERT into an Iceberg table with spark.comet.iceberg.write.enabled=true
and iceberg-spark-runtime 1.12.0 on the classpath. The plan has no CometIcebergWrite, and the
WARN above is the only signal.

Expected behavior

Native writes work on Iceberg 1.12. The 3-arg read(manifest, io, specsById) is public on every
version Comet supports, and passing the table's specs() (or a single-entry map for the stamped
specId) would work on 1.5.2 through 1.12.

Additional context

Comet 1.1.0, Spark 3.5.6. Works on Iceberg 1.11.0. The Spark profiles in spark/pom.xml pin
1.5.2 / 1.8.1 / 1.10.0 / 1.11.0, so no CI profile covers 1.12; adding one would have caught this.
Related: #4871, #5308, #5649, #7040.

Dominant language
Scala
Stars
1.3k
Forks
405
Avg merge
2d 21h
Merged PRs (30d)
329

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion-comet

All issues in apache/datafusion-comet

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.