Hacktoberfest 2026:维护者为十月标记出来的 issue,仍然开放、适合新手。 浏览 Hacktoberfest issue

[C++][Parquet] Plaintext-footer files written with AES_GCM_CTR_V1 record AES_GCM_V1 as the encryption algorithm and cannot be read

未关闭 适合新手
#51,669 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

@YusefSyed 已经在做这个了。

开始于 2026年10月2日。

  • #51693 来自 @YusefSyed —— 未关闭

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
88/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
cpp
领域
databases

调研方向

从 cpp/src/parquet/metadata.cc 中的 FileMetaDataBuilder::FileMetaDataBuilderImpl::Finish 开始,然后检查 cpp/src/parquet/encryption/write_configurations_test.cc 中的加密配置。为使用明文 footer 的 AES_GCM_CTR_V1 添加覆盖测试,并验证加密文件能够成功完成往返。完成的标准是元数据记录文件的实际算法,同时 footer 签名仍能通过验证。

由索引模型根据 Issue 内容生成。

描述

Type: bug
Describe the bug, including details regarding any error messages, version, and platform.

A Parquet file written with modular encryption in plaintext-footer mode and the AES_GCM_CTR_V1 algorithm cannot be read back, by Arrow or by any other reader. The writer encrypts the pages with AES-CTR, as it should, but records AES_GCM_V1 in FileMetaData.encryption_algorithm. A reader takes the algorithm from that field, tries to open the CTR pages as GCM modules, and fails authentication.

Nothing is reported at write time, so the file looks fine until someone tries to read an encrypted column.

Reproduction
import pyarrow as pa, pyarrow.parquet as pq, pyarrow.parquet.encryption as pe

key = b"0123456789012345"
table = pa.table({"a": [1, 2, 3]})
for algo in ("AES_GCM_V1", "AES_GCM_CTR_V1"):
    props = pe.create_encryption_properties(
        key, plaintext_footer=True, encryption_algorithm=algo)
    pq.write_table(table, "t.parquet", encryption_properties=props)
    try:
        pq.read_table(
            "t.parquet",
            decryption_properties=pe.create_decryption_properties(key))
        print(pa.__version__, algo, "ok")
    except OSError as e:
        print(pa.__version__, algo, "FAILED:", e)

Output:

25.0.1 AES_GCM_V1 ok
25.0.1 AES_GCM_CTR_V1 FAILED: Failed decryption finalization

The same two algorithms with an encrypted footer (plaintext_footer=False) both round-trip, as does AES_GCM_V1 with a plaintext footer. Only the combination of a plaintext footer and AES_GCM_CTR_V1 fails.

Cause

FileMetaDataBuilder::FileMetaDataBuilderImpl::Finish in cpp/src/parquet/metadata.cc builds the footer's algorithm for plaintext-footer mode like this (current main):

// if plaintext footer, set footer signing algorithm
auto file_encryption_properties = properties_->file_encryption_properties();
if (file_encryption_properties && !file_encryption_properties->encrypted_footer()) {
  EncryptionAlgorithm signing_algorithm;
  EncryptionAlgorithm algo = file_encryption_properties->algorithm();
  signing_algorithm.aad.aad_file_unique = algo.aad.aad_file_unique;
  signing_algorithm.aad.supply_aad_prefix = algo.aad.supply_aad_prefix;
  if (!algo.aad.supply_aad_prefix) {
    signing_algorithm.aad.aad_prefix = algo.aad.aad_prefix;
  }
  signing_algorithm.algorithm = ParquetCipher::AES_GCM_V1;

  metadata_->__set_encryption_algorithm(ToThrift(signing_algorithm));

The hard-coded AES_GCM_V1 treats FileMetaData.encryption_algorithm as the algorithm of the footer signature, which is indeed always GCM. But the field is the file's encryption algorithm. parquet.thrift describes it as:

/**
 * Encryption algorithm. This field is set only in encrypted files
 * with plaintext footer. Files with encrypted footer store algorithm id
 * in FileCryptoMetaData structure.
 */
8: optional EncryptionAlgorithm encryption_algorithm

and the reader uses it that way: SerializedFile::ParseMetaDataOfEncryptedFileWithPlaintextFooter in cpp/src/parquet/file_reader.cc passes file_metadata_->encryption_algorithm().algorithm to the InternalFileDecryptor, which then builds the page decryptors from it.

parquet-java writes the file's actual algorithm in this field (ParquetFileWriter.serializeFooter: parquetMetadata.setEncryption_algorithm(fileEncryptor.getEncryptionAlgorithm())).

Evidence that the pages are fine and only the field is wrong

Taking a file produced by the reproduction above, changing the one byte that selects the EncryptionAlgorithm union member in the footer from AES_GCM_V1 to AES_GCM_CTR_V1, and recomputing the footer signature for the changed bytes, gives a file that pyarrow 25.0.1 reads correctly, with footer verification enabled and the data equal to what was written. So the reader needs no change, and the pages in affected files are valid CTR modules.

Suggested fix

Write the file's algorithm instead of the constant:

signing_algorithm.algorithm = algo.algorithm;

The signature itself does not depend on this value: both the writer and FileMetaData::VerifySignature use the GCM cipher for the footer regardless of the file's algorithm (metadata = true).

The existing encryption configurations in cpp/src/parquet/encryption/write_configurations_test.cc use AES_GCM_CTR_V1 only with an encrypted footer and a plaintext footer only with AES_GCM_V1, which is why the round-trip tests do not catch this. A configuration combining the two would cover it.

Files already written with this combination stay unreadable after the fix, since their footers name the wrong algorithm.

Version and platform
  • pyarrow 25.0.1 (PyPI wheel, manylinux_2_28 x86_64), Python 3.14.4, Linux x86_64 (Fedora 44).
  • The hard-coded line is present on main as of commit fcac3dfd8e (2026-09-30).
Component(s)

C++, Parquet

主要语言
C++
星标
17.2k
派生
4.4k
平均合并
3 天 22 小时
30 天内合并 PR
80

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

apache/arrow 的其他 Issue

查看 apache/arrow 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。