Effect ↔ Rust interop: Schema findings, concrete asks, and an open question

Start by reading the linked #1550, #1556, and #1578, then review the seven repros around Schema.toCodecJson, JSON Schema import/export, and SchemaAST traversal. This issue needs a maintainer-selected scope first; done would require one narrowly defined ask with an agreed API and acceptance criteria.

TL;DR

• We built an Effect 4 ↔ Rust (wasm + Node-API) boundary: one contract owner (Effect Schema or Rust serde), a compiler that emits both sides, and a generated Context.Service with explicit wasm/native Layers. Spec: #1550; code: #1556, #1578.
• Walking live Schema ASTs for cross-language codegen exposed seven Effect-side gaps. We re-verified each one against [email protected] (npm, Bun 1.4.2). They are listed below with minimal repros.
• We'd value your view on the asks, and we'd also like to know how much of this, if any, the Effect team would want to be involved in (see the last section). This is a question, not a package proposal.

Asks

All repros were typechecked and run against [email protected]. Where 4.0.0 behaviour is documented or intentional, the ask says so. These asks matter for codegen because a Rust/serde side has to match Effect's accepted input set exactly. Any value that one side accepts and the other rejects is a cross-language contract bug.

1. Strict canonical input for toCodecJson(Schema.BigInt)

const Big = Schema.toCodecJson(Schema.BigInt)
Schema.decodeUnknownSync(Big)("00")  // 0n   (also "01" -> 1n, "-0" -> 0n)
// "+1", " 1", "1e3", "0x10" are rejected

• Expected: "00", "01" and "-0" are rejected, or there is an opt-in canonical mode. Actual: they are accepted and normalised, so decode → encode is not the identity on the wire.
• Why it matters: the Rust side (and content hashing) needs one canonical decimal lexeme per value. Today we add an encoded-side filter, /^(0|-?[1-9][0-9]*)$/, in our own derived codec.
• Duplicates: none found (searched effect + effect-smol for "leading zero", "BigIntFromString", "BigInt toCodecJson").

2. Strict RFC 3339 input for toCodecJson(Schema.DateTimeUtc)

const At = Schema.toCodecJson(Schema.DateTimeUtc)
decode("2026-10-02T00:00:00.1234Z") // accepted, truncated to .123Z
decode("2026-02-30T00:00:00.000Z") // accepted, normalised to 2026-03-02
decode("2026-10-02")               // accepted (date-only)
decode("2026-10-02T00:00:00")      // accepted (no offset; read as UTC)
decode("October 2, 2026")          // accepted

• Expected: an RFC 3339 grammar with an explicit offset, and rejection of invalid calendar dates. Ideally, sub-millisecond input would be rejected rather than truncated, at least as an option. Actual: the parsing is Date-like and lenient.
• Why: chrono/serde reject all five inputs, so the two sides disagree on acceptance. Silent truncation also loses information before any check can see it, because the decoded DateTime.Utc no longer carries the original text.
• Related: #6790 covers format: "date-time" on export, a different issue.

3. Per-schema excess-property policy

const Outer = Schema.Struct({ strict: Inner, open: Schema.Struct({ a: Schema.String }) })
decode(Outer)({ strict: { a: "x", extra: 1 }, open: { a: "x", extra: 1 } })  // both stripped
decode(Outer, { onExcessProperty: "error" })(…)                                 // both rejected
// closing one struct via StructWithRest(Inner, [Record(String, Never)]) rejects { a: "x" } too

• Expected: a way to close one struct and leave another open. v3 offered this through the parseOptions annotation, and v4's .annotate({ parseOptions }) is accepted but has no effect. Actual: the policy can only be set globally at the call site.
• Why: Rust contracts default to deny_unknown_fields, with per-type opt-outs. Mixed policies are common: closed records with an open metadata map.
• Related: #3027 (v3, schema-level ParseOptions), effect-smol#2499.

4. JSON Schema export: report approximations, and emit exact safe-integer bounds

Schema.toJsonSchemaDocument(Schema.Int).schema            // { type: "integer" }  (2**53 is rejected by Effect)
Schema.toJsonSchemaDocument(Schema.String.check(Schema.isPattern(/^[a-f0-9]{64}$/))).schema // { type: "string" }
// /^[a-z]+$/i and /^[a-z]+$/iu are also omitted; only /…/u survives

• Context: we know #8482 (closing #8358) made these mappings intentionally looser and tracks approximation internally. That is the right default for preliminary validation.
• Ask (a): export isInt as type: "integer" with minimum: -(2**53-1) and maximum: 2**53-1. This is exact rather than approximate, and it was the first isInt item in #8358's checklist. Today the PR marks isInt as approximate instead.
• Ask (b): make approximation status visible. For example, an option onApproximation: "error", or a list of approximate JSON pointers returned with the document.
• Why: a generator that reads the exported document cannot tell which constraints were dropped. In one experiment, Schema.Int.check(Schema.isGreaterThan(0)) exported without a maximum, and the generated serde type then accepted 2^53 and i64::MAX, which Effect rejects. Separately, a non-u digest regex was omitted, so generated Rust accepted invalid digests. We now walk the AST instead (ask 7), but other JSON Schema consumers hit the same problem silently.

5. JSON Schema import: an extension-keyword hook and an unknown-keyword policy

const s = SchemaRepresentation.fromJsonSchemaDocument(
  JsonSchema.fromSchemaDraft2020_12({ type: "string", "x-wire-format": "u64-decimal" }))
decode(s)("not a number")  // accepted: unknown keyword silently ignored (documented)
const u = …({ type: "integer", format: "uint32", minimum: 0 }); decode(u)(2**32)  // accepted

• Expected: a hook that maps a keyword or node to a Schema, for example onKeyword: (key, value, node) => Schema.Top | undefined, plus an option to reject unknown keywords. Actual: onEnter can only rewrite JSON Schema → JSON Schema. It cannot attach a codec or declaration, and unknown keywords are ignored.
• Why: schemars (Rust) emits format: "uint32" without bounds, and Rust-owned contracts need vendor keywords for u64-as-decimal, millisecond timestamps and brands. In our bakeoff, the native importer reached 0 disagreements on the 35 portable vectors only after onEnter normalisation. It still disagreed on 6/27 (Effect-owned) and 13/27 (Rust-owned) extension vectors, because the extension keywords were dropped.
• Related: #7410 (closed by rejecting unsupported validation keywords: same fail-closed spirit), #7418.

6. Optional-key omission in toCodecJson (low priority, documented behaviour)

Schema.encodeSync(Schema.toCodecJson(Schema.Struct({ a: Schema.optional(Schema.String) })))({ a: undefined })
// { a: null }

• Ask: an option that encodes own-property undefined at a schema-declared optional key by omitting the key. The current mapping is documented, and #8491 aligned the export with it.
• Why: serde's Option<T> + skip_serializing_if cannot distinguish an omitted key from null, but our Patch<T> can. Encoding undefined as null changes a two-state field into the three-state shape on the wire.

7. A public structural AST map/rebuild

const ast = Schema.Struct({ a: Schema.BigInt }).ast
ast.encodingChecks          // public in 4.0.0
ast.recur(f)                // exists at runtime, TS2339 (stripped from .d.ts)
SchemaAST.replaceContext(…) // exists at runtime, TS2339; same for replaceChecks/appendChecks/annotate

• Expected: a public SchemaAST.mapChildren(ast, f), or a typed recur, that preserves annotations, checks, encoding, context and encodingChecks. Actual: a codec deriver (for example, "replace every BigInt leaf with a canonical decimal codec") has to call each node constructor by hand. That hand-rolled rebuild breaks whenever a constructor gains a field, and encodingChecks was added during the RCs.
• Related: #3382 (v3: "expose internals used in JSONSchema"), #8052.

Checked and dropped: we hit self._build is not a function when a consumer on rc.113 used a Layer generated against rc.118. We could not reduce it: Layers built with rc.118 and provided to rc.113 or 4.0.0 apps worked in both directions in a minimal repro. So it is a question below, not an ask.

Motivation

• Rust libraries stay plain Rust, with no JS types, for Rust callers. Effect callers get Services, Effects, typed errors, Sinks and Streams.
• One authoring owner per contract. Generation must preserve validation, presence (missing/null/value) and integer precision, not just types. Contracts that cannot be represented fail generation; they are never weakened.
• The application explicitly chooses wasm, native or a subprocess. There is no detection or fallback, and a failed load never changes execution policy.

Design sketch (shortened from implemented code; planned parts are marked)

Rust core: no wasm-bindgen, napi or Effect dependency.

pub fn hash(bytes: &[u8]) -> Digest { let mut h = Hasher::new(); h.update(bytes); h.finish() }
#[cfg_attr(feature = "contract", derive(serde::Deserialize), serde(try_from = "String"))]
pub struct Digest(pub [u8; 32]);   // wire form: validated "sha256:<hex>"

Adapter crate: one attribute per export; the macro emits wasm-bindgen/napi glue plus an export manifest.

#[effect_rust::export]
pub fn hash(bytes: effect_rust::Bytes) -> Digest { core::hash(&bytes) }
#[effect_rust::export(input_stream, returns = "Digest")]   // becomes a Sink on the Effect side
pub fn hasher() -> core::Hasher { core::Hasher::new() }

Effect-owned contract: plain Schema plus a few namespaced annotations (decided; the implemented fixture still uses an older Wire.* vocabulary).

const Deployment = Schema.Struct({
  replicas: Schema.Int.check(Schema.isBetween({ minimum: 0, maximum: 100 })), // width inferred: u8; optional pin
  counter: Schema.BigInt.check(Schema.isBetweenBigInt({ minimum: 0n, maximum: 2n ** 64n - 1n })), // u64, decimal on JSON
  at: Schema.DateTimeUtc.annotate({ [EffectRust.timestampPrecision]: "millis" }),
  host: Schema.String.check(Schema.isPattern(/^[a-z][a-z0-9-]*$/u)).annotate({ identifier: "HostName" }),
  note: Schema.optional(Schema.String),                     // schema-aware omission on the wire (decided)
  owner: Schema.optionalKey(Schema.NullOr(Schema.String)),  // three-state; Patch<String> in Rust
})
const codec = ContractJson.codec(Deployment)  // transports: ContractJson / Borsh / Columns

Rust-owned contract: serde stays authoritative, and schemars metadata enters the same compiler.

#[effect_rust::contract]
#[derive(Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "camelCase")]
pub struct Order { #[wire(u64)] pub id: u64, pub sku: Sku, #[wire(timestamp_millis)] pub placed_at: DateTime<Utc>, pub note: Patch<String> }

Effect consumer: a generated Service class, one TaggedError per Rust enum with a reason union.

const program = Effect.gen(function* () {
  const core = yield* ContentAddressCore
  const digest = yield* core.hash(bytes)
  const streamed = yield* Stream.run(chunks, core.hasher())
  return { digest, streamed }
}).pipe(Effect.catchReason("TreeError", "NotFound", (r) => Effect.succeed(r.path)))

App integrator: explicit Layer choice, with Init failing only at Layer construction.

Effect.provide(program, ContentAddressCore.layerWasm.node())    // or .bun() / .browser() / .worker()
Effect.provide(program, ContentAddressCore.layerNative.node())  // desktop opt-in
// browsers default to an external .wasm asset; inline base64 is opt-in

Evidence

All numbers are scoped experiments on shared, often loaded hosts. They are directional, not benchmarks.

Study	Result	Caveat
Generator bakeoff (Typify, schemafy, quicktype, Effect importer, @xschemadev)	No stack passed all 6 must-haves in either direction. A custom compiler sharing one IR had 0 disagreements on 35 portable, 27 extension and 7 regex-flag vectors, for both owners; 16 non-portable inputs were rejected with path + remedy; 34 generated files were byte-identical across reruns	Fixture-sized; regex grammar is an allowlist
Effect importer, Rust-owned direction	fromJsonSchemaDocument → toCodeDocument works as the Rust→Effect backend	Needs onEnter width normalisation; extensions lost (ask 5)
Annotation-first authoring	Plain-Schema rewrite of all 7 fixture roots: 137/137 vectors in TS and in generated Rust, identical lowered IR	Prototype, not yet the production cutover
Generated Rust shape	6 shapes, 0 disagreements on 125 vectors; chose plain serde + validating newtypes	—
u64/i64 over JSON	Decimal-string/bigint was the fastest lossless JSON option on Node and Bun	Codec throughput, not end-to-end calls
Binary bulk formats (11)	All lossless; an emitted Borsh codec is in the fastest tier (+1.8 KB gzip JS); reflective borsh-js was 13–46× slower	workerd: arrow-js encode fails
Several cores per app	One combined wasm is 35% smaller than three (brotli 21.3 vs 32.5 KB) and initialises faster	3 small cores
Cancellation	10k create/cancel cycles per runtime/backend, 0 leaked Rust handles, flat linear memory	Cooperative only; no CPU preemption
Panics	Lexical instance factory + scheduler fence contained every wasm trap (Node/Bun/Chromium/workerd); native panic=unwind contained; panic=abort → SIGABRT	workerd delayed-GC regression filed: workerd#7598
Pilot 1 (public, byte hashing)	Net 656 handwritten lines including the algorithms and a 133-line app facade; wasm 1.32 MB, native addon 2.97 MB	Shared foundation is 9,013 LOC (compiler 2,818)
Internal app pilot A (image/byte processing, two cores)	Aggregate wasm 34–37% smaller than the previous pair; warm init 6–10 ms vs 14–21 ms; manual glue 120 → 123 lines	First init with an inline base64 loader was much worse (Chromium 342 vs 85 ms), so browsers now default to an external asset. Mixed Effect RCs produced self._build is not a function
Internal app pilot B (stateful fuzzy matcher, Rust-owned contracts)	41/41 parity; 10k create/drop, 0 leaks	Generated JSON/Effect boundary 3.2× (Bun) / 5.4× (Node) slower than the raw binding on 20–45 µs calls, even with codecs hoisted. Next: profile, then a typed direct transport for wasm/native; resource exports planned

What we'd like to know

1. Involvement. How would you like to engage with this, if at all? (a) feedback only, (b) collaboration on the Effect-side pieces (asks 1–7, maybe a shared contract-codec story), or (c) interest in the Effect team owning some part long-term? Each of these, including (a), is a fine answer.
2. Are check representation ids and payloads (effect/schema/isInt, isBetween, isPattern {source, flags}) a stable public contract we can key a compiler on? We reject any check without a representation.
3. Should strict, canonical boundary codecs (asks 1, 2 and 6) live in Effect as an opt-in, or should they stay downstream on top of public AST APIs (ask 7)?
4. Is a generated Context.Service class with static layerWasm/layerNative constructors idiomatic v4, or would you expose codecs + a factory and leave the tag to the app?
5. v3 had a runtime version-mismatch guard (#3308, #1479). Is there a v4 equivalent we should use to fail fast on mixed copies?

Deep dive: schema compiler

• The front-end walks the live SchemaAST, never exported JSON Schema, because export is lossy by design (ask 4). Admission is an allowlist: primitives, Struct/Record/Tuple/Union/Suspend, literal discriminators, and checks identified by representation id. Opaque refinements, arbitrary transformations and runtime defaults are rejected with path + remedy.
• Integers: bounded Schema.Int is admitted directly. The width is inferred from the interval (exact intervals → u8…i32, subranges → validating newtypes), with an optional pin annotation. A width change is a layout change and bumps the frame version. BigInt needs explicit bounds that fit u64/i64.
• Strings: patterns use a reviewed cross-engine grammar with u/iu only, with differential tests against JS u-mode and the Rust regex crate. Lengths count code points (isBetweenCodePoints). Lone UTF-16 surrogates cannot round-trip through Rust UTF-8, so such metadata stays Effect-owned.
• Presence: optionalKey is "missing only". Schema.optional uses schema-aware omission. optionalKey(NullOr(T)) ⇔ Patch<T> (missing/null/value).
• Errors: one TaggedError per Rust error enum, with a reason union, so catchReason maps 1:1 to Rust matching. Panics are defects, never domain errors.
• Canonical JSON: duplicate keys rejected, depth ≤ 128, tag-first output (any order accepted on input). This is a contract-specific profile, not JCS.
• Our codec deriver rebuilds AST nodes by hand to swap BigInt/DateTime leaves (ask 7).

Deep dive: runtime semantics (cancellation, panics, ownership)

• Each Layer gets its own lexical wasm instance from a shared compiled WebAssembly.Module. Stock wasm-bindgen glue is a module singleton, so we generate a factory. In workerd, there is one runtime per isolate.
• Cancellation is cooperative: Effect interruption → AbortSignal → Rust Abortable, and host capabilities declare whether they are abortable or settle-only. 10k create/cancel cycles showed no leaked handles.
• Panic containment: a wasm trap poisons that instance's generation, every pending Effect fails as a defect, and the default policy rebuilds the instance. Failed operations are never replayed. A panic during a wasm-bindgen mutable borrow leaves the instance unusable even if later calls succeed, so we retire it. Native code requires panic=unwind, which is checked at build time through an attestation symbol. Abort, OOM and double panics cannot be contained in-process: use subprocess isolation. A new Layer does not unload a native addon.
• Ownership: bytes are copied into Rust-owned memory for async/native jobs, with no zero-copy promises. Stream input/output has a byte budget, but downstream buffering can exceed it. A ~64 KiB chunk default avoids Effect scheduling overhead on tiny chunks.
• Planned: resource exports (stateful Rust objects as scoped Effect resources, prompted by pilot B) and a typed direct transport for wasm/native calls, with JSON reserved for process and storage boundaries.

Deep dive: packaging and composition

• Package conditions: workerd, bun, node, browser, default. workerd needs precompiled wasm. Browsers depend on CSP and default to an external .wasm asset (pilot A: inline base64 made first init 4× slower).
• Several cores → one generated app crate → one wasm, with one shared runtime Layer that each per-core Service depends on.
• The core/adapter split keeps Rust reuse free of JS. Rust's orphan rule pushes contract traits into feature-gated core metadata or adapter newtypes.
• Effect peers must be one exact cohort. A consumer on rc.113 with a generator on rc.118 failed at runtime (see question 5).
• Native targets: Linux x64/arm64 and macOS arm64 tested; Windows untested. Bun has no dedicated JS Worker constructor in our matrix.
• Build: direct Cargo plus the packaging scripts work. Our Buck2/Nix integration is a choice, not a requirement, but there is no turnkey public CLI yet.

Deep dive: complexity budget (measured 2026-10-02, physical LOC)

Area	LOC
Effect runtime (instances, jobs, poisoning, Layers)	1,239
Wire/schema/Borsh support	993
Schema compiler, importers, emitters	2,818
Rust support + macros (incl. 444 in-file test lines)	2,896
Build rules + product/service generators	1,060
Foundation total (37 files, incl. 7 lines of root exports)	9,013

• The smallest adopter case, add(i32, i32), needs 11 Rust lines (core + adapter) and generates 74 lines. A counted reference configuration with manifests is ~117 lines.
• Pilot 1: 661 lines added / 5 deleted. Generated output: contract crate + schemas, 1,288 lines; service + codecs, 498 lines.
• Breakdown: inherent costs are widths, presence, panic=unwind and wasm toolchains. Chosen costs are the adapter split, the generated Service and Borsh. Incidental tooling debt is the schemars bridge, Cargo patching and generated-TS admission. Of these, only the compiler (2,818) would shrink with Effect-side help (asks 4, 5, 7).

Posted on behalf of @schickling

field	value
agent_identity	unknown
session	dev3.01a0e83f
agent_persona	generalist
agent_supervisor	unavailable
agent_tool	OMP
agent_tool_version	18.4.10
agent_runtime	OMP 18.4.10
agent_model	anthropic/claude-opus-5-5
worktree	unknown
tooling_profile	dotfiles@794d88a