Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

docs: fix stale and missing native write details in iceberg-writes.md and iceberg.md

Open Beginner friendly
#6,147 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
75/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
rust, scala, spark
Domain
documentation

Research direction

The issue points to specific lines in docs/source/user-guide/latest/iceberg-writes.md and iceberg.md. Start by reading those files to understand the current text. Check the referenced pull requests (#5840, #6114) and the test suite CometIcebergWriteActionSuite for context on what needs updating. Verify the behavior of the spark.comet.exec.localTableScan.enabled configuration. The goal is to correct the listed inaccuracies and add the missing details.

Written by the indexing model from the issue text.

Description

area:Iceberg area:writer documentation requires-triage
What is the problem the feature request solves?

The Iceberg user guide has errors and gaps about native writes:

  • docs/source/user-guide/latest/iceberg-writes.md:273-276 says float/double partition directories use Rust's shortest representation (f=1 where iceberg-java writes f=1.0). Since #5840 they use Java's Double.toString rules and match iceberg-java.
  • iceberg-writes.md:316-319 says the parity suite compares committed manifests byte-for-byte. CometIcebergWriteActionSuite compares aggregated readable_metrics (min/max, counts), not manifest bytes.
  • iceberg-writes.md does not say that writes whose input is a local relation, such as INSERT ... VALUES and df.writeTo(...).append() on a local DataFrame, need spark.comet.exec.localTableScan.enabled (default false). Without it they silently use iceberg-java even with both write flags on.
  • iceberg-writes.md does not list #6114 (dictionary page kept for high-cardinality columns) among the accepted divergences.
  • docs/source/user-guide/latest/iceberg.md:184 lists "Iceberg writes (reads are accelerated, writes use Spark)" as unsupported. Native writes exist behind spark.comet.iceberg.write.enabled; that page should point to iceberg-writes.md.
Describe the potential solution

Fix the four points above. The "experimental, disabled by default" wording will need another pass when the defaults change (#5644).

Additional context

Found in an audit of the native Iceberg write path before enabling it by default. Part of #5649.

Dominant language
Scala
Stars
1.3k
Forks
377
Avg merge
2d 11h
Merged PRs (30d)
243

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion-comet

All issues in apache/datafusion-comet

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.