Schema: a public way to derive a canonical codec, as `toCodecJson` and `toCodecStringTree` do

Aperta
  • Qualcosa non va?
#10,457 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@effect-bot ci sta già lavorando.

Dal 7/11/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

Needs Proposal schema
What is the problem this feature would solve?

This is one focused use case split out of #8690, as requested there. It is related to ask 7 ("A public structural AST map/rebuild"), but it covers more.

We write a Firestore integration for Effect. Firestore stores a fixed set of values: null, booleans, numbers, strings, timestamps, bytes, geo points, document references, arrays and maps. That makes it a canonical format in the same way JSON is one. So we derive toCodecFirestore(schema) the way Effect derives toCodecJson(schema):

  • A declaration is looked up by a toCodecFirestore annotation, then toCodecJson, then toCodec, the same fallback toCodecStringTree uses.
  • Date, DateTimeUtc and Uint8Array map to Firestore's own timestamp and bytes types instead of strings.
  • Unknown, Any and ObjectKeyword accept only values Firestore can store.
  • Keywords, objects, unions, arrays and suspends are handled as toCodecJson handles them.

The step function is ours, but the machinery around it is only available internally. toCodecFirestore is a copy of the internal makeCodecDerivation in internal/schema/toCodec.ts plus our own step function. Building it needs these exports, all of which exist at runtime but are @internal and stripped from the .d.ts:

Internal API What we use it for
SchemaAST.applyToSelfOrLastLinkEncodingIdempotent Run the step on a node or on its last link's target, memoized
SchemaAST.replaceEncoding Give a declaration or keyword its canonical link
SchemaAST.mapLink Recurse into the target of a declaration's toCodec* link
SchemaAST.replaceContextLastLink Move the context, minus the constructor default, onto the last link
SchemaAST.parameterFromString Read object keys back from strings
ast.recur(f, onKey?) Recurse into Objects, Arrays, Union and Suspend
ast.toCodecJson() / ast.toCodecStringTree() on keyword nodes Reuse Effect's canonical form for undefined, literals, bigint and symbols

Today we type these by hand behind a cast and pin effect to an exact version. This is fragile:

  • A patch release can change or remove any of them without notice. That is fair, since they are internal.
  • Our copy drifts from Effect's. makeCodecDerivation now derives each body once per context owner (getContextOwner + replaceContext) so that shared schema references stay shared. Our copy predates that change and doesn't get it.
  • Every library that wants its own format (a database's value model, a wire format, a config tree) has to reimplement the same few dozen lines against internals.
What is the feature you are proposing to solve the problem?

Make the derivation machinery public, so that a library supplies only the step function and Effect owns the rest. Roughly:

// SchemaAST
export function makeCodecDerivation(
  step: (ast: AST, recur: (ast: AST) => AST) => AST,
  options?: { readonly stopAt?: (link: Link) => boolean }
): (ast: AST) => AST

The step function also needs a few public building blocks:

  • replaceEncoding(ast, encoding) and mapLink(link, f)
  • parameterFromString
  • A typed ast.recur(f, onKey?), or a SchemaAST.recur(ast, f, onKey?) function. This overlaps with ask 7 of #8690.
  • Public access to the canonical JSON and StringTree forms of a keyword node, for example SchemaAST.keywordToCodecJson(ast) and SchemaAST.keywordToCodecStringTree(ast), or simply public toCodecJson() / toCodecStringTree() methods

With these, toCodecFirestore comes down to:

import { Schema, SchemaAST } from "effect"

const toCodecFirestoreAST = SchemaAST.makeCodecDerivation((ast, recur) => {
  switch (ast._tag) {
    case "Declaration": /* look up toCodecFirestore ?? toCodecJson ?? toCodec, mapLink(link, recur) */
    case "Unknown":
    case "Any":
    case "ObjectKeyword":
      return SchemaAST.replaceEncoding(ast, [unknownToFirestore])
    case "Objects":
      return SchemaAST.recur(ast, recur, SchemaAST.parameterFromString)
    // …
  }
  return ast
})

export const toCodecFirestore = <S extends Schema.Constraint>(schema: S) =>
  Schema.make(toCodecFirestoreAST(schema.ast), { schema })

Effect's own toCodecJson, toCodecStringTree and toCodecGoJson would be built on the same public API, so the API gets tested by Effect itself and keeps up with fixes such as the context-owner sharing.

A smaller first step would also help us: publishing the individual helpers listed above, even without makeCodecDerivation.

What alternatives have you considered?
  • Use toCodecJson and convert JSON to Firestore values afterwards. This loses Firestore's own types. A Date becomes a string instead of a timestamp, and bytes become base64. Reading back can't tell a timestamp-as-string from a real string.
  • Build the codec through the public node constructors. This needs a hand-written rebuild of every node kind. As #8690 ask 7 notes, it breaks whenever a constructor gains a field (encodingChecks was added during the RCs). It also still can't reach parameterFromString or the keywords' canonical forms.
  • Keep the internal helpers behind a cast with effect pinned exactly (what we do now). It works, but consumers can't take Effect patch releases without us re-checking the internals first.
Lingua principale
TypeScript
Stelle
16.7k
Fork
808
Merge medio
12h 17m
PR unite (30g)
805

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Effect-TS/effect

Tutte le issue di Effect-TS/effect

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.