Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Truncate transform diverges from Iceberg Java's TruncateUtil for widths above 2^30, and panics in debug builds at the integer minimum

オープン
  • 何か問題がありますか？
#3,141 コメント 1 件 リアクション 2 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
78/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
rust
領域
databases

調査の方向性

crates/iceberg/src/transform/truncate.rs から始めて、truncate_i32、truncate_i64、truncate_decimal_i128 を調査します。リストされている大きな幅と整数の最小値のケースを debug ビルドと release ビルドで再現し、その後、結果を Apache Iceberg Java TruncateUtil の例と比較します。報告された境界値についてカーネルが Java と一致し、debug ビルドで panic が発生しなければ完了です。

索引モデルが issue の本文から書いたものです。

説明

Apache Iceberg Rust version

main @ 3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7

Describe the bug

Truncate::transform in crates/iceberg/src/transform/truncate.rs does not reproduce Iceberg Java's TruncateUtil arithmetic. There are two distinct problems.

1. truncate_i32 diverges from Java for widths above 2^30

fn truncate_i32(v: i32, width: i32) -> i32 {
    v - v.rem_euclid(width)
}

Java computes the positive remainder with an expression that is allowed to overflow:

// org.apache.iceberg.util.TruncateUtil
public static int truncateInt(int width, int value) {
  return value - (((value % width) + width) % width);
}

(value % width) + width overflows int whenever value % width > Integer.MAX_VALUE - width, which is reachable for any width > 2^30. rem_euclid never overflows, so the two produce different partition values. Only positive values with a large width are affected; negative values and widths at or below 2^30 agree.

Truncate.get(int width) in Iceberg Java only validates width > 0, so these widths are accepted by the spec. Whether Java's overflow is desirable is beside the point — it is the value iceberg-java writes into the partition tuple, so a Rust writer that computes something else produces a table the Java reader partitions differently.

2. Overflow panics in debug builds at the integer minimum

fn truncate_i64(v: i64, width: i64) -> i64 {
    v - (((v % width) + width) % width)
}

For v = i64::MIN, width = 1000 the remainder is 192 and v - 192 underflows. With debug_assertions on this panics (attempt to subtract with overflow); in a release build it wraps and matches Java exactly. truncate_i32 has the same shape at i32::MIN, and truncate_decimal_i128 uses the same unchecked subtraction (unreachable for a 38-digit Iceberg decimal, but the same pattern).

A panic here aborts the write task, so a debug or test build of any downstream writer fails on data that Java handles.

To Reproduce

Values below were produced by running TruncateUtil from apache-iceberg 1.11.0 on JDK 17 against the Rust expressions:

width value Java TruncateUtil.truncateInt iceberg-rust truncate_i32
2000000000 1500000000 -2000000000 0
2147483647 2147483646 -2147483647 0
1073741824 1500000000 1073741824 1073741824
1000 2147483647 2147483000 2147483000

For problem 2, in a debug build:

let t = create_transform_function(&Transform::Truncate(1000)).unwrap();
t.transform(Arc::new(Int64Array::from(vec![i64::MIN]))).unwrap(); // panics
Expected behavior

Both kernels bit-identical to Java in debug and release. Wrapping arithmetic gives that:

fn truncate_i32(v: i32, w: i32) -> i32 {
    v.wrapping_sub((v % w).wrapping_add(w) % w)
}

fn truncate_i64(v: i64, w: i64) -> i64 {
    v.wrapping_sub((v % w).wrapping_add(w) % w)
}

That fixes both problems at once: it reproduces Java's overflow for large widths and removes the debug panic. The alternative for problem 1 is to reject width > 2^30 at construction, which overlaps with #2473, but that would reject specs iceberg-java accepts.

Willingness to contribute

I would be willing to contribute a fix for this bug with guidance from the Iceberg community.

How this was found. Apache DataFusion Comet is adding native kernels for Iceberg's Spark system functions (apache/datafusion-comet#5638). On a partitioned write the sort key comes from those kernels (Java semantics) while the partition values the clustered writer groups by come from iceberg-rust's transforms, and the clustered writer fails at runtime with The input is not sorted! Cannot write to partition that was previously closed when the two disagree. Cross-checking the two implementations over boundary inputs is what surfaced this.

This report was drafted with LLM assistance (Claude Code); the divergences were verified against Iceberg Java on a JVM.

主要言語
Rust
スター
1.4k
フォーク
574
平均マージ
1日 19時間
マージ済み PR（30日）
66

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/iceberg-rust のほかの issue

apache/iceberg-rust の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。