Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

V1 manifest lists written by Java lose added/existing/deleted file counts

Open Beginner friendly
  • Is something wrong?
#3,445 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

@kacper-li-airspace-intelligence is already working on this.

Since Oct 14, 2026.

  • #3447 by @kacper-li-airspace-intelligence — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
data

Research direction

Start with the ManifestFileV1 struct and MANIFEST_LIST_AVRO_SCHEMA_V1 in the manifest list code, and compare them with ManifestFileV2, which already has aliases for both count names. Run the reproduction in the issue, which parses a V1 manifest list with Java's field names via ManifestList::parse_with_version(_, FormatVersion::V1). Done means the three file counts parse as Some and a test uses Java's V1 names. The issue says this depends on #3224 (V1 reads against the writer schema), so check that first.

Written by the indexing model from the issue text.

Description

Apache Iceberg Rust version

main (1270e329b)

Describe the bug

Java writes V1 manifest lists with the field names added_files_count, existing_files_count and deleted_files_count (V1Metadata.MANIFEST_LIST_SCHEMA, ManifestFile), which are the names in the spec. ManifestFileV1 and MANIFEST_LIST_AVRO_SCHEMA_V1 only know the older added_data_files_count names, so ManifestList::parse_with_version(_, FormatVersion::V1) returns None for all three counts. Fields with the same name in both, such as added_rows_count, parse correctly. Reading the same bytes as V2 returns the counts, because ManifestFileV2 has aliases for both names.

The spec treats a null count as non-zero, so scan planning stays correct, but the counts are lost. FastAppendAction::existing_manifest copies these entries into the next manifest list, so a fast append on such a table writes the nulls back.

To Reproduce

Write one manifest list entry with Java's V1 field names and parse it:

use apache_avro::types::Record;
use apache_avro::{Schema, Writer};
use iceberg::spec::{FormatVersion, ManifestList};

let opt = |n: &str, t: &str| format!(r#"{{"name":"{n}","type":["null","{t}"],"default":null}}"#);
let schema = Schema::parse_str(&format!(
    r#"{{"type":"record","name":"manifest_file","fields":[
    {{"name":"manifest_path","type":"string"}},{{"name":"manifest_length","type":"long"}},
    {{"name":"partition_spec_id","type":"int"}},{{"name":"added_snapshot_id","type":"long"}},
    {},{},{},{},{},{},{}]}}"#,
    opt("added_files_count", "int"), opt("existing_files_count", "int"), opt("deleted_files_count", "int"),
    opt("added_rows_count", "long"), opt("existing_rows_count", "long"), opt("deleted_rows_count", "long"),
    opt("key_metadata", "bytes"),
)).unwrap();
let mut writer = Writer::new(&schema, Vec::new()).unwrap();
let mut record = Record::new(&schema).unwrap();
record.put("manifest_path", "s3://t/m.avro");
record.put("manifest_length", 1i64);
record.put("partition_spec_id", 0i32);
record.put("added_snapshot_id", 1i64);
record.put("added_files_count", Some(3i32));
record.put("existing_files_count", Some(0i32));
record.put("deleted_files_count", Some(0i32));
record.put("added_rows_count", Some(9i64));
record.put("existing_rows_count", Some(0i64));
record.put("deleted_rows_count", Some(0i64));
record.put("key_metadata", None::<Vec<u8>>);
writer.append_value(record).unwrap();
let bs = writer.into_inner().unwrap();

let list = ManifestList::parse_with_version(&bs, FormatVersion::V1).unwrap();
let entry = &list.entries()[0];
// added_files_count, existing_files_count, deleted_files_count: None
// added_rows_count: Some(9)
Expected behavior

V1 parsing returns Some(3), Some(0) and Some(0) for the three file counts. Once V1 reads against the writer schema (#3224), adding #[serde(alias = ...)] for added_files_count, existing_files_count and deleted_files_count to ManifestFileV1, as ManifestFileV2 already does, fixes it. The fix should come with a test that uses Java's V1 names.

Willingness to contribute

I can contribute a fix for this bug independently

Related
Dominant language
Rust
Stars
1.4k
Forks
592
Avg merge
5d 12h
Merged PRs (30d)
59

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-rust

All issues in apache/iceberg-rust

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.