Editions: gate extension dtypes and support crate-declared families (json)

Aberta
  • Algo errado?
#548 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
45/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
java, rust
Domínio
backend, data

Direção de pesquisa

Start with the edition catalog and VortexWriter.create, then compare the current encoding-only guard with the extension-dtype behavior described in vortex-json/src/editions.rs. Update docs/compatibility.md, ADR 0023, and CLAUDE.md alongside the implementation, including the decision about whether EditionFamily stays closed. Done means schemas with disabled extension dtypes are rejected, the json family and vortex.json behavior are covered, and the built-in JSON decoder rejects non-UTF-8 data.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

enhancement

Rust editions gate extension dtypes as well as encodings, and edition families are no longer a fixed set: optional crates declare their own. vortex-java models neither.

Rust behavior

  • An edition lists its members by component kind, and an extension dtype is one of those kinds (EditionMember::dtype(..)). A writer with an edition policy may emit only members of enabled families, so a schema with an extension dtype outside them is refused.
  • Families are declared per crate. vortex-json/src/editions.rs declares the json family (origin vortex-json) with one draft edition, json2026.08.0, adding the vortex.json dtype. vortex_json::initialize(session) registers the dtype and enables the family together. Being outside core reflects that a reader built without vortex-json cannot resolve vortex.json.

vortex-java today

  1. The edition guard (WriteOptions#editions(), ADR 0023) checks array encodings only, so withEdition(..) lets a schema carry vortex.json or any custom extension id (now easy to register, #530/#547). docs/compatibility.md already notes the catalog models encodings only.
  2. EditionFamily is a closed enum (CORE/PREVIEW/ZSTD), and CLAUDE.md justifies this because a private family carries no real guarantee. Rust now has crate-declared families (json), so the enum is missing one, and the closed-world rationale no longer matches the reference.

To decide

  • Extend the Editions catalog with extension-dtype members and enforce them in the writer's edition guard at VortexWriter.create.
  • Model json (and decide between keeping EditionFamily closed with known Rust families added, and opening it the way Rust does). Update ADR 0023 / CLAUDE.md accordingly.
  • Related: a built-in vortex.json ExtensionDecoder<String> (UTF-8 storage, validate rejecting non-Utf8), so JSON files open without allowUnknown() after #547.

Refs #530

Linguagem predominante
Java
Estrelas
3
Forks
2
Merge médio
14h 59min
PRs com merge (30d)
137

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de dfa1/vortex-java

Todas as issues de dfa1/vortex-java

Issues semelhantes

Mais issues de Java

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.