feat(writer): write vortex.union (declared in Rust's unstable2026.10.0 edition)

Abierto
  • ¿Algo no está bien?
#660 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
38/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
java, rust
Área
backend, databases

Línea de trabajo

Read #442 for the union wire layout, then inspect UnionEncodingEncoder registration in writer.encode and the cascade logic in vortex-compressor/src/compressor/cascade.rs. Check edition-family handling against ADR 0023 and the unstable declarations in vortex/editions/unstable/unstable2026.10.0.toml. Done means writing and cascading unions with unstable-edition gating, correcting docs/compatibility.md, and adding cross-language coverage; the issue lists a Rust-native harness as the first testing option.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Write side of #442. #442 deferred the writer because vortex.union was in no Rust edition ("Revisit when an edition declares it"). That is no longer true: Rust's unstable2026.10.0 edition (vortex/editions/unstable/unstable2026.10.0.toml, vortex-edition/src/declarations/unstable/mod.rs) lists vortex.union. docs/compatibility.md (lines 39, 150) still says "in no edition" and needs correcting as part of this.

Scope

  • UnionEncodingEncoder in writer.encode, registered in WriteRegistry.Builder#registerDefaults(): type ids (U8, nullable iff the union is) + one row-aligned child per variant, no metadata, no buffers (Rust rejects either). Layout per #442.
  • Cascading: compress type_ids and each child independently and rebuild the union, as vortex-compressor/src/compressor/cascade.rs does.
  • Edition gating: Rust now has an unstable family; EditionFamily is CORE/PREVIEW/ZSTD. Decide whether to add it (ADR 0023 says closed enum mirroring Rust's families, so probably yes) and make a default write never emit vortex.union.
  • Update docs/compatibility.md, AGENTS.md if the edition family changes, CHANGELOG.

Testing (vortex-jni cannot help: vortex-arrow bails on DataType::Union)

Ideas, cheapest first:

  1. Rust-native cargo harness (as for the Pco encoder): a small Rust program linking vortex-array/vortex-file that (a) reads a Java-written file and dumps the array tree + values, and (b) builds a UnionArray and writes it with the unstable edition, so Java reads real Rust output. This also gives the #442 decoder its first real-producer coverage.
  2. Rust-side vtx/vortex-tui browse on a Java-written file, as a quick manual smoke check.
  3. Byte-compare node trees: write the same logical union through the Rust harness and Java, diff the ArrayNode trees (the replicate-JNI-on-size-gaps approach).
  4. Reuse Rust's own test vectors in vortex-array/src/arrays/union/tests as unit fixtures.
  5. Keep the hand-built reader tests from #442; add random unions via a seeded @MethodSource (variant counts 1..256, non-consecutive type ids, null tags, nested union children).

Open question: is the unstable edition's vortex.union wire layout final? It makes no stability promise, so changes may follow.

Lenguaje dominante
Java
Estrellas
3
Forks
2
Merge medio
10 h 41 min
PR fusionados (30 d)
136

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dfa1/vortex-java

Todos los issues de dfa1/vortex-java

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.