Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

bug: gzip table metadata is neither written nor detected as gzip

Aperta
  • Qualcosa non va?
#1,013 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@LuciferYang ci sta già lavorando.

Dal 14/10/2026.

  • #1014 di @LuciferYang — aperta

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
66/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
data

Direzione di ricerca

Start in src/iceberg/table_metadata.cc, at Codec::FromFileName, where find_last_of(kTableMetadataFileSuffix) searches for any single character rather than the suffix, and at TableMetadataUtil::Write, which never compresses. The existing GZipDecompressor in util/gzip_internal.h is the model for a matching GZipCompressor. Done means a gzip-configured table writes gzip metadata that is read back correctly, and a name without .metadata.json is rejected. Both halves need to land together.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

With write.metadata.compression-codec=gzip, iceberg-cpp names new metadata files <version>-<uuid>.gz.metadata.json, the same layout Iceberg Java uses, but the round trip is broken on both sides:

  • TableMetadataUtil::Write never compresses. It writes plain JSON into the .gz.metadata.json file, so Java (or any reader that trusts the name) fails to inflate it.
  • Codec::FromFileName misdetects the codec. It calls find_last_of(".metadata.json"), which searches for any single character of that set, not the suffix. For 00001-abc.gz.metadata.json it lands on the final n, the .gz check fails, and the file is read as uncompressed. A gzip file written by Java under that name therefore fails to parse.

The two bugs mask each other inside iceberg-cpp: a file it writes is plain JSON and is read back as plain JSON, so its own round trip happens to work.

Root cause

src/iceberg/table_metadata.cc:

auto pos = file_name.find_last_of(kTableMetadataFileSuffix);  // character-set search

and TableMetadataUtil::Write(io, location, metadata) serializes and calls io.WriteFile with no codec step. Only a GZipDecompressor exists in util/gzip_internal.h.

Java's TableMetadataParser.Codec.fromFileName uses lastIndexOf(".metadata.json"), and internalWrite wraps the stream in GZIPOutputStream when that codec is gzip.

Impact

A table configured with gzip metadata compression is not readable across engines in either direction: Java cannot read what iceberg-cpp writes, and iceberg-cpp cannot read what Java writes. The default codec is none, so tables without that property are unaffected.

The find_last_of bug also means FromFileName accepts names with no .metadata.json at all (e.g. foo.txt is classified as uncompressed), while Java rejects them.

Proposed fix

Use rfind in FromFileName, and compress in Write when the location's codec is gzip, with a small GZipCompressor next to the existing GZipDecompressor. Both halves have to land together: fixing only the read side makes iceberg-cpp try to inflate the plain JSON it writes today.

Lingua principale
C++
Stelle
228
Fork
136
Merge medio
5g 19h
PR unite (30g)
53

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/iceberg-cpp

Tutte le issue di apache/iceberg-cpp

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.