Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

bug: gzip table metadata is neither written nor detected as gzip

Aberta
  • Algo errado?
#1,013 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 2 dias

@LuciferYang já está trabalhando nisso.

Desde 14/10/2026.

  • #1014 de @LuciferYang — aberto

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
66/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
cpp
Domínio
data

Direção de pesquisa

Start in src/iceberg/table_metadata.cc, at Codec::FromFileName, where find_last_of(kTableMetadataFileSuffix) searches for any single character rather than the suffix, and at TableMetadataUtil::Write, which never compresses. The existing GZipDecompressor in util/gzip_internal.h is the model for a matching GZipCompressor. Done means a gzip-configured table writes gzip metadata that is read back correctly, and a name without .metadata.json is rejected. Both halves need to land together.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Summary

With write.metadata.compression-codec=gzip, iceberg-cpp names new metadata files <version>-<uuid>.gz.metadata.json, the same layout Iceberg Java uses, but the round trip is broken on both sides:

  • TableMetadataUtil::Write never compresses. It writes plain JSON into the .gz.metadata.json file, so Java (or any reader that trusts the name) fails to inflate it.
  • Codec::FromFileName misdetects the codec. It calls find_last_of(".metadata.json"), which searches for any single character of that set, not the suffix. For 00001-abc.gz.metadata.json it lands on the final n, the .gz check fails, and the file is read as uncompressed. A gzip file written by Java under that name therefore fails to parse.

The two bugs mask each other inside iceberg-cpp: a file it writes is plain JSON and is read back as plain JSON, so its own round trip happens to work.

Root cause

src/iceberg/table_metadata.cc:

auto pos = file_name.find_last_of(kTableMetadataFileSuffix);  // character-set search

and TableMetadataUtil::Write(io, location, metadata) serializes and calls io.WriteFile with no codec step. Only a GZipDecompressor exists in util/gzip_internal.h.

Java's TableMetadataParser.Codec.fromFileName uses lastIndexOf(".metadata.json"), and internalWrite wraps the stream in GZIPOutputStream when that codec is gzip.

Impact

A table configured with gzip metadata compression is not readable across engines in either direction: Java cannot read what iceberg-cpp writes, and iceberg-cpp cannot read what Java writes. The default codec is none, so tables without that property are unaffected.

The find_last_of bug also means FromFileName accepts names with no .metadata.json at all (e.g. foo.txt is classified as uncompressed), while Java rejects them.

Proposed fix

Use rfind in FromFileName, and compress in Write when the location's codec is gzip, with a small GZipCompressor next to the existing GZipDecompressor. Both halves have to land together: fixing only the read side makes iceberg-cpp try to inflate the plain JSON it writes today.

Linguagem predominante
C++
Estrelas
226
Forks
133
Merge médio
6d 2h
PRs com merge (30d)
49

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de apache/iceberg-cpp

Todas as issues de apache/iceberg-cpp

Issues semelhantes

Mais issues de C++

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.