Editions: gate extension dtypes and support crate-declared families (json)

Ouverte
  • Quelque chose ne va pas ?
#548 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
45/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
java, rust
Domaine
backend, data

Piste de recherche

Start with the edition catalog and VortexWriter.create, then compare the current encoding-only guard with the extension-dtype behavior described in vortex-json/src/editions.rs. Update docs/compatibility.md, ADR 0023, and CLAUDE.md alongside the implementation, including the decision about whether EditionFamily stays closed. Done means schemas with disabled extension dtypes are rejected, the json family and vortex.json behavior are covered, and the built-in JSON decoder rejects non-UTF-8 data.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

enhancement

Rust editions gate extension dtypes as well as encodings, and edition families are no longer a fixed set: optional crates declare their own. vortex-java models neither.

Rust behavior

  • An edition lists its members by component kind, and an extension dtype is one of those kinds (EditionMember::dtype(..)). A writer with an edition policy may emit only members of enabled families, so a schema with an extension dtype outside them is refused.
  • Families are declared per crate. vortex-json/src/editions.rs declares the json family (origin vortex-json) with one draft edition, json2026.08.0, adding the vortex.json dtype. vortex_json::initialize(session) registers the dtype and enables the family together. Being outside core reflects that a reader built without vortex-json cannot resolve vortex.json.

vortex-java today

  1. The edition guard (WriteOptions#editions(), ADR 0023) checks array encodings only, so withEdition(..) lets a schema carry vortex.json or any custom extension id (now easy to register, #530/#547). docs/compatibility.md already notes the catalog models encodings only.
  2. EditionFamily is a closed enum (CORE/PREVIEW/ZSTD), and CLAUDE.md justifies this because a private family carries no real guarantee. Rust now has crate-declared families (json), so the enum is missing one, and the closed-world rationale no longer matches the reference.

To decide

  • Extend the Editions catalog with extension-dtype members and enforce them in the writer's edition guard at VortexWriter.create.
  • Model json (and decide between keeping EditionFamily closed with known Rust families added, and opening it the way Rust does). Update ADR 0023 / CLAUDE.md accordingly.
  • Related: a built-in vortex.json ExtensionDecoder<String> (UTF-8 storage, validate rejecting non-Utf8), so JSON files open without allowUnknown() after #547.

Refs #530

Langage dominant
Java
Étoiles
3
Forks
2
Merge moyen
9 h 47 min
PR mergées (30 j)
128

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de dfa1/vortex-java

Toutes les issues de dfa1/vortex-java

Issues similaires

Plus d'issues Java

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.