Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Feature] Support writing ARRAY<BLOB> fields

Abierto
  • ¿Algo no está bien?
#389 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cpp, java, python
Área
data-engineering, databases

Línea de trabajo

Start with the ARRAY reader work from #387/#388, then trace BlobFormatWriter, schema validation, the data-evolution write path, and append/data-evolution compaction. Use the BLOB file specification as the format reference. Done means validation, null and descriptor handling, writing, compaction, round-trip tests, and Java/Python compatibility tests all pass.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement
Search before asking
  • I searched in the issues and found nothing similar.
Motivation

Java Paimon (apache/paimon#8181) and PyPaimon can write top-level ARRAY<BLOB> columns to dedicated .blob files. #387 (PR #388) adds read-only ARRAY<BLOB> support to Paimon C++, but C++ table creation, writes, and append compaction still reject these fields. As a result, C++ users cannot write ordered collections of multimodal objects, such as the frames or images of one sample, to a Paimon table.

Solution

Add Java-compatible write support for top-level ARRAY<BLOB> fields declared with blob-field:

  • Schema validation: Match Java SchemaValidation and ColumnDirectiveUtils:
    • allow ARRAY<BLOB> only as a top-level field declared with blob-field;
    • reject it for blob-descriptor-field, partition keys, and tables without data evolution;
    • reject BLOB elements nested more deeply, such as ARRAY<ARRAY<BLOB>>.
  • Blob format writer: Extend BlobFormatWriter to encode each ARRAY<BLOB> row as the nested payload defined in the BLOB file spec. The payload contains:
    • array magic 1094861634;
    • version 1;
    • element count;
    • concatenated element data;
    • Delta-Varint element length index;
    • index length.
  • Null handling: Distinguish null arrays (outer length -1), empty arrays (element count 0), null elements (element length -1), and empty elements (element length 0).
  • Element sources: Support both inline raw bytes and BlobDescriptor elements. Descriptor elements are materialized from their referenced storage when written, as Java does.
  • Write path: Route ARRAY<BLOB> fields to dedicated blob files in the data-evolution write path, and remove the write-side rejection for them.
  • Compaction: Support ARRAY<BLOB> in append and data-evolution compaction, including placeholder (-2) handling across field groups.
  • Tests:
    • add unit tests for payload encoding, null/empty cases, and error handling;
    • add round-trip tests with the C++ reader from #387;
    • add Java/Python cross-compatibility tests, with C++ writing and Java/Python reading.
Anything else?
  • Depends on #387 / #388 for the reader side.
  • MAP<K, BLOB> write support and ARRAY<BLOB> in primary-key managed BLOB storage (#204) are out of scope and can be tracked separately.
  • No public API changes are expected. The storage format follows the existing Java spec.
Are you willing to submit a PR?
  • I'm willing to submit a PR!
Lenguaje dominante
C++
Estrellas
65
Forks
29
Merge medio
2 d 5 h
PR fusionados (30 d)
77

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/paimon-cpp

Todos los issues de apache/paimon-cpp

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.