Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

[Feature] Support writing ARRAY<BLOB> fields

未关闭
  • 哪里不对？
#389 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
cpp, java, python
领域
data-engineering, databases

调研方向

Start with the ARRAY reader work from #387/#388, then trace BlobFormatWriter, schema validation, the data-evolution write path, and append/data-evolution compaction. Use the BLOB file specification as the format reference. Done means validation, null and descriptor handling, writing, compaction, round-trip tests, and Java/Python compatibility tests all pass.

由索引模型根据 Issue 内容生成。

描述

enhancement
Search before asking
  • I searched in the issues and found nothing similar.
Motivation

Java Paimon (apache/paimon#8181) and PyPaimon can write top-level ARRAY<BLOB> columns to dedicated .blob files. #387 (PR #388) adds read-only ARRAY<BLOB> support to Paimon C++, but C++ table creation, writes, and append compaction still reject these fields. As a result, C++ users cannot write ordered collections of multimodal objects, such as the frames or images of one sample, to a Paimon table.

Solution

Add Java-compatible write support for top-level ARRAY<BLOB> fields declared with blob-field:

  • Schema validation: Match Java SchemaValidation and ColumnDirectiveUtils:
    • allow ARRAY<BLOB> only as a top-level field declared with blob-field;
    • reject it for blob-descriptor-field, partition keys, and tables without data evolution;
    • reject BLOB elements nested more deeply, such as ARRAY<ARRAY<BLOB>>.
  • Blob format writer: Extend BlobFormatWriter to encode each ARRAY<BLOB> row as the nested payload defined in the BLOB file spec. The payload contains:
    • array magic 1094861634;
    • version 1;
    • element count;
    • concatenated element data;
    • Delta-Varint element length index;
    • index length.
  • Null handling: Distinguish null arrays (outer length -1), empty arrays (element count 0), null elements (element length -1), and empty elements (element length 0).
  • Element sources: Support both inline raw bytes and BlobDescriptor elements. Descriptor elements are materialized from their referenced storage when written, as Java does.
  • Write path: Route ARRAY<BLOB> fields to dedicated blob files in the data-evolution write path, and remove the write-side rejection for them.
  • Compaction: Support ARRAY<BLOB> in append and data-evolution compaction, including placeholder (-2) handling across field groups.
  • Tests:
    • add unit tests for payload encoding, null/empty cases, and error handling;
    • add round-trip tests with the C++ reader from #387;
    • add Java/Python cross-compatibility tests, with C++ writing and Java/Python reading.
Anything else?
  • Depends on #387 / #388 for the reader side.
  • MAP<K, BLOB> write support and ARRAY<BLOB> in primary-key managed BLOB storage (#204) are out of scope and can be tracked separately.
  • No public API changes are expected. The storage format follows the existing Java spec.
Are you willing to submit a PR?
  • I'm willing to submit a PR!
主要语言
C++
星标
65
派生
29
平均合并
2 天 5 小时
30 天内合并 PR
77

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

apache/paimon-cpp 的其他 Issue

查看 apache/paimon-cpp 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。