Hacktoberfest 2026：維護者為十月標記出來的 issue，仍然開放、適合新手。 瀏覽 Hacktoberfest issue

[Feature] Support writing ARRAY<BLOB> fields

未關閉
  • 哪裡不對？
#389 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 1 天內回覆

還沒有人認領這個 Issue。

評估

難度
5/5
預估耗時
一週以上
新手友好度
35/100
Issue 類型
功能
描述清晰度
基本清楚
活躍度
活躍
技術堆疊
cpp, java, python
領域
data-engineering, databases

研究方向

Start with the ARRAY reader work from #387/#388, then trace BlobFormatWriter, schema validation, the data-evolution write path, and append/data-evolution compaction. Use the BLOB file specification as the format reference. Done means validation, null and descriptor handling, writing, compaction, round-trip tests, and Java/Python compatibility tests all pass.

由索引模型根據 Issue 內容生成。

描述

enhancement
Search before asking
  • I searched in the issues and found nothing similar.
Motivation

Java Paimon (apache/paimon#8181) and PyPaimon can write top-level ARRAY<BLOB> columns to dedicated .blob files. #387 (PR #388) adds read-only ARRAY<BLOB> support to Paimon C++, but C++ table creation, writes, and append compaction still reject these fields. As a result, C++ users cannot write ordered collections of multimodal objects, such as the frames or images of one sample, to a Paimon table.

Solution

Add Java-compatible write support for top-level ARRAY<BLOB> fields declared with blob-field:

  • Schema validation: Match Java SchemaValidation and ColumnDirectiveUtils:
    • allow ARRAY<BLOB> only as a top-level field declared with blob-field;
    • reject it for blob-descriptor-field, partition keys, and tables without data evolution;
    • reject BLOB elements nested more deeply, such as ARRAY<ARRAY<BLOB>>.
  • Blob format writer: Extend BlobFormatWriter to encode each ARRAY<BLOB> row as the nested payload defined in the BLOB file spec. The payload contains:
    • array magic 1094861634;
    • version 1;
    • element count;
    • concatenated element data;
    • Delta-Varint element length index;
    • index length.
  • Null handling: Distinguish null arrays (outer length -1), empty arrays (element count 0), null elements (element length -1), and empty elements (element length 0).
  • Element sources: Support both inline raw bytes and BlobDescriptor elements. Descriptor elements are materialized from their referenced storage when written, as Java does.
  • Write path: Route ARRAY<BLOB> fields to dedicated blob files in the data-evolution write path, and remove the write-side rejection for them.
  • Compaction: Support ARRAY<BLOB> in append and data-evolution compaction, including placeholder (-2) handling across field groups.
  • Tests:
    • add unit tests for payload encoding, null/empty cases, and error handling;
    • add round-trip tests with the C++ reader from #387;
    • add Java/Python cross-compatibility tests, with C++ writing and Java/Python reading.
Anything else?
  • Depends on #387 / #388 for the reader side.
  • MAP<K, BLOB> write support and ARRAY<BLOB> in primary-key managed BLOB storage (#204) are out of scope and can be tracked separately.
  • No public API changes are expected. The storage format follows the existing Java spec.
Are you willing to submit a PR?
  • I'm willing to submit a PR!
主要語言
C++
星號
65
分支
29
平均合併
2 天 5 小時
30 天內合併 PR
77

環境準備

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

apache/paimon-cpp 的其他 Issue

查看 apache/paimon-cpp 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。