feat(writer): write vortex.union (declared in Rust's unstable2026.10.0 edition)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
Read #442 for the union wire layout, then inspect UnionEncodingEncoder registration in writer.encode and the cascade logic in vortex-compressor/src/compressor/cascade.rs. Check edition-family handling against ADR 0023 and the unstable declarations in vortex/editions/unstable/unstable2026.10.0.toml. Done means writing and cascading unions with unstable-edition gating, correcting docs/compatibility.md, and adding cross-language coverage; the issue lists a Rust-native harness as the first testing option.
由索引模型根据 Issue 内容生成。
描述
Write side of #442. #442 deferred the writer because vortex.union was in no Rust edition ("Revisit when an edition declares it"). That is no longer true: Rust's unstable2026.10.0 edition (vortex/editions/unstable/unstable2026.10.0.toml, vortex-edition/src/declarations/unstable/mod.rs) lists vortex.union. docs/compatibility.md (lines 39, 150) still says "in no edition" and needs correcting as part of this.
Scope
UnionEncodingEncoderinwriter.encode, registered inWriteRegistry.Builder#registerDefaults(): type ids (U8, nullable iff the union is) + one row-aligned child per variant, no metadata, no buffers (Rust rejects either). Layout per #442.- Cascading: compress
type_idsand each child independently and rebuild the union, asvortex-compressor/src/compressor/cascade.rsdoes. - Edition gating: Rust now has an
unstablefamily;EditionFamilyisCORE/PREVIEW/ZSTD. Decide whether to add it (ADR 0023 says closed enum mirroring Rust's families, so probably yes) and make a default write never emitvortex.union. - Update
docs/compatibility.md, AGENTS.md if the edition family changes, CHANGELOG.
Testing (vortex-jni cannot help: vortex-arrow bails on DataType::Union)
Ideas, cheapest first:
- Rust-native cargo harness (as for the Pco encoder): a small Rust program linking
vortex-array/vortex-filethat (a) reads a Java-written file and dumps the array tree + values, and (b) builds aUnionArrayand writes it with the unstable edition, so Java reads real Rust output. This also gives the #442 decoder its first real-producer coverage. - Rust-side
vtx/vortex-tuibrowse on a Java-written file, as a quick manual smoke check. - Byte-compare node trees: write the same logical union through the Rust harness and Java, diff the
ArrayNodetrees (the replicate-JNI-on-size-gaps approach). - Reuse Rust's own test vectors in
vortex-array/src/arrays/union/testsas unit fixtures. - Keep the hand-built reader tests from #442; add random unions via a seeded
@MethodSource(variant counts 1..256, non-consecutive type ids, null tags, nested union children).
Open question: is the unstable edition's vortex.union wire layout final? It makes no stability promise, so changes may follow.
- 主要语言
- Java
- 星标
- 3
- 派生
- 2
- 平均合并
- 9 小时 55 分钟
- 30 天内合并 PR
- 126
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
dfa1/vortex-java 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 66/100
dfa1/vortex-java#636 ·
维护者通常 1 天内回复
-
bug
难度 4/5 3-5 天 新手友好度 35/100
dfa1/vortex-java#657 · 1 条评论 ·
维护者通常 1 天内回复
-
bug
难度 3/5 1-2 天 新手友好度 70/100
dfa1/vortex-java#655 ·
维护者通常 1 天内回复
-
perf
难度 4/5 3-5 天 新手友好度 30/100
dfa1/vortex-java#651 ·
维护者通常 1 天内回复
-
Writer: string columns written as per-chunk dicts where Rust uses its dict layout (up to 2x larger)未关闭perf
难度 4/5 3-5 天 新手友好度 30/100
dfa1/vortex-java#650 ·
维护者通常 1 天内回复
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
scylladb/scylla-cdc-source-connector#402 ·
维护者通常 1 天内回复
-
enhancement
难度 2/5 1-3 小时 新手友好度 74/100
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 64/100
维护者通常 1 天内回复
-
enhancement good first issue
难度 2/5 1-3 小时 新手友好度 72/100
ahmedmmordi/store-app#8 ·
-
jhipster heroku fails on heroku git:remote when reusing an existing app outside a git repository未关闭area: triage theme: undefined
难度 2/5 1-3 小时 新手友好度 76/100
jhipster/generator-jhipster#36018 ·
维护者通常 1 天内回复