Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[C++][Parquet] Plaintext-footer files written with AES_GCM_CTR_V1 record AES_GCM_V1 as the encryption algorithm and cannot be read

Aperta Adatta ai principianti
  • Qualcosa non va?
#51,669 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
88/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
databases

Direzione di ricerca

Inizia in cpp/src/parquet/metadata.cc, in FileMetaDataBuilder::FileMetaDataBuilderImpl::Finish, quindi esamina le configurazioni di crittografia in cpp/src/parquet/encryption/write_configurations_test.cc. Aggiungi la copertura per AES_GCM_CTR_V1 con un footer in testo normale e verifica che il file crittografato completi correttamente il round-trip. Il lavoro è completato quando i metadati registrano l'algoritmo effettivo del file e la firma del footer continua a essere verificata.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Type: bug
Describe the bug, including details regarding any error messages, version, and platform.

A Parquet file written with modular encryption in plaintext-footer mode and the AES_GCM_CTR_V1 algorithm cannot be read back, by Arrow or by any other reader. The writer encrypts the pages with AES-CTR, as it should, but records AES_GCM_V1 in FileMetaData.encryption_algorithm. A reader takes the algorithm from that field, tries to open the CTR pages as GCM modules, and fails authentication.

Nothing is reported at write time, so the file looks fine until someone tries to read an encrypted column.

Reproduction
import pyarrow as pa, pyarrow.parquet as pq, pyarrow.parquet.encryption as pe

key = b"0123456789012345"
table = pa.table({"a": [1, 2, 3]})
for algo in ("AES_GCM_V1", "AES_GCM_CTR_V1"):
    props = pe.create_encryption_properties(
        key, plaintext_footer=True, encryption_algorithm=algo)
    pq.write_table(table, "t.parquet", encryption_properties=props)
    try:
        pq.read_table(
            "t.parquet",
            decryption_properties=pe.create_decryption_properties(key))
        print(pa.__version__, algo, "ok")
    except OSError as e:
        print(pa.__version__, algo, "FAILED:", e)

Output:

25.0.1 AES_GCM_V1 ok
25.0.1 AES_GCM_CTR_V1 FAILED: Failed decryption finalization

The same two algorithms with an encrypted footer (plaintext_footer=False) both round-trip, as does AES_GCM_V1 with a plaintext footer. Only the combination of a plaintext footer and AES_GCM_CTR_V1 fails.

Cause

FileMetaDataBuilder::FileMetaDataBuilderImpl::Finish in cpp/src/parquet/metadata.cc builds the footer's algorithm for plaintext-footer mode like this (current main):

// if plaintext footer, set footer signing algorithm
auto file_encryption_properties = properties_->file_encryption_properties();
if (file_encryption_properties && !file_encryption_properties->encrypted_footer()) {
  EncryptionAlgorithm signing_algorithm;
  EncryptionAlgorithm algo = file_encryption_properties->algorithm();
  signing_algorithm.aad.aad_file_unique = algo.aad.aad_file_unique;
  signing_algorithm.aad.supply_aad_prefix = algo.aad.supply_aad_prefix;
  if (!algo.aad.supply_aad_prefix) {
    signing_algorithm.aad.aad_prefix = algo.aad.aad_prefix;
  }
  signing_algorithm.algorithm = ParquetCipher::AES_GCM_V1;

  metadata_->__set_encryption_algorithm(ToThrift(signing_algorithm));

The hard-coded AES_GCM_V1 treats FileMetaData.encryption_algorithm as the algorithm of the footer signature, which is indeed always GCM. But the field is the file's encryption algorithm. parquet.thrift describes it as:

/**
 * Encryption algorithm. This field is set only in encrypted files
 * with plaintext footer. Files with encrypted footer store algorithm id
 * in FileCryptoMetaData structure.
 */
8: optional EncryptionAlgorithm encryption_algorithm

and the reader uses it that way: SerializedFile::ParseMetaDataOfEncryptedFileWithPlaintextFooter in cpp/src/parquet/file_reader.cc passes file_metadata_->encryption_algorithm().algorithm to the InternalFileDecryptor, which then builds the page decryptors from it.

parquet-java writes the file's actual algorithm in this field (ParquetFileWriter.serializeFooter: parquetMetadata.setEncryption_algorithm(fileEncryptor.getEncryptionAlgorithm())).

Evidence that the pages are fine and only the field is wrong

Taking a file produced by the reproduction above, changing the one byte that selects the EncryptionAlgorithm union member in the footer from AES_GCM_V1 to AES_GCM_CTR_V1, and recomputing the footer signature for the changed bytes, gives a file that pyarrow 25.0.1 reads correctly, with footer verification enabled and the data equal to what was written. So the reader needs no change, and the pages in affected files are valid CTR modules.

Suggested fix

Write the file's algorithm instead of the constant:

signing_algorithm.algorithm = algo.algorithm;

The signature itself does not depend on this value: both the writer and FileMetaData::VerifySignature use the GCM cipher for the footer regardless of the file's algorithm (metadata = true).

The existing encryption configurations in cpp/src/parquet/encryption/write_configurations_test.cc use AES_GCM_CTR_V1 only with an encrypted footer and a plaintext footer only with AES_GCM_V1, which is why the round-trip tests do not catch this. A configuration combining the two would cover it.

Files already written with this combination stay unreadable after the fix, since their footers name the wrong algorithm.

Version and platform
  • pyarrow 25.0.1 (PyPI wheel, manylinux_2_28 x86_64), Python 3.14.4, Linux x86_64 (Fedora 44).
  • The hard-coded line is present on main as of commit fcac3dfd8e (2026-09-30).
Component(s)

C++, Parquet

Lingua principale
C++
Stelle
17.2k
Fork
4.3k
Merge medio
4g 15h
PR unite (30g)
107

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/arrow

Tutte le issue di apache/arrow

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.