Editions: gate extension dtypes and support crate-declared families (json)

未关闭
#548 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
java, rust
领域
backend, data

调研方向

Start with the edition catalog and VortexWriter.create, then compare the current encoding-only guard with the extension-dtype behavior described in vortex-json/src/editions.rs. Update docs/compatibility.md, ADR 0023, and CLAUDE.md alongside the implementation, including the decision about whether EditionFamily stays closed. Done means schemas with disabled extension dtypes are rejected, the json family and vortex.json behavior are covered, and the built-in JSON decoder rejects non-UTF-8 data.

由索引模型根据 Issue 内容生成。

描述

enhancement

Rust editions gate extension dtypes as well as encodings, and edition families are no longer a fixed set: optional crates declare their own. vortex-java models neither.

Rust behavior

  • An edition lists its members by component kind, and an extension dtype is one of those kinds (EditionMember::dtype(..)). A writer with an edition policy may emit only members of enabled families, so a schema with an extension dtype outside them is refused.
  • Families are declared per crate. vortex-json/src/editions.rs declares the json family (origin vortex-json) with one draft edition, json2026.08.0, adding the vortex.json dtype. vortex_json::initialize(session) registers the dtype and enables the family together. Being outside core reflects that a reader built without vortex-json cannot resolve vortex.json.

vortex-java today

  1. The edition guard (WriteOptions#editions(), ADR 0023) checks array encodings only, so withEdition(..) lets a schema carry vortex.json or any custom extension id (now easy to register, #530/#547). docs/compatibility.md already notes the catalog models encodings only.
  2. EditionFamily is a closed enum (CORE/PREVIEW/ZSTD), and CLAUDE.md justifies this because a private family carries no real guarantee. Rust now has crate-declared families (json), so the enum is missing one, and the closed-world rationale no longer matches the reference.

To decide

  • Extend the Editions catalog with extension-dtype members and enforce them in the writer's edition guard at VortexWriter.create.
  • Model json (and decide between keeping EditionFamily closed with known Rust families added, and opening it the way Rust does). Update ADR 0023 / CLAUDE.md accordingly.
  • Related: a built-in vortex.json ExtensionDecoder<String> (UTF-8 storage, validate rejecting non-Utf8), so JSON files open without allowUnknown() after #547.

Refs #530

主要语言
Java
星标
3
派生
2
平均合并
14 小时 59 分钟
30 天内合并 PR
137

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

dfa1/vortex-java 的其他 Issue

查看 dfa1/vortex-java 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。