Cluster: a CBOR codec, chosen as `codec: cbor`

Open
  • Is something wrong?
#266 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
45/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
distributed-systems

Research direction

Start with tellus/src/cluster/codec.rs to understand the erased deserializer interface, then inspect tests/cluster.rs and the test_config entry point. The proposed scope also calls for codec unit tests, a CBOR cluster test recipe, and documentation updates covering configuration, rolling upgrades, and Schema::VERSION. Done means the CBOR codec works through those tests and the docs explain its compatibility tradeoffs and restart requirement.

Written by the indexing model from the issue text.

Description

The second provided codec, as agreed in #262. Plan first, no PR before you confirm, and no hurry: after 0.6, whenever it fits.

What

A Cbor codec next to Postcard in cluster::codec, named "cbor" at the handshake and chosen as codec: cbor (CodecConfig::Cbor) or config.codec = Arc::new(Cbor). A cluster still uses one codec: another one is refused as CodecMismatch, so the switch is a cluster restart, as the docs say (813-817). Frames stay postcard, only payloads change, and postcard stays the default.

What it buys, and what it costs

CBOR is self-describing, a struct is a map keyed by field name and a variant is named, so a shape changed without a version bump never decodes wrong. Measured with ciborium:

within one VERSION postcard CBOR
fields swapped, a variant inserted or reordered decodes wrong correct
a field replaced, or its type changed decodes wrong (u32 7 as i32 is -4) decode error, or correct where the value fits the new type
a field added with #[serde(default)] dead letter decodes both ways
a field removed, a new variant sent to old code dead letter dead letter (a removal decodes at old code only with a default there)

So additive changes need neither a bump nor an encode override, your "CBOR stays an option on top for additive changes" from #261. The price is size, 2 to 4 times postcard (6 to 24 bytes for a three-field order, 1,501 to 3,302 for a hundred readings); CPU the PR measures with just bench-cluster.

The crate

Codec::decode hands the visitor an erased serde::Deserializer (codec.rs 14-46), so the crate must expose one. ciborium, which persistence uses, does not: its deserializer is private, and Value is none either. So minicbor-serde 0.7 (Deserializer::new(&[u8]), decoder().position() for the trailing-bytes check; two crates behind cluster), or cbor4ii 1.2 (one crate, but its slice reader exposes no position, so the check needs a reader of our own). Both encode structs as maps keyed by name. I'd take minicbor-serde.

Scope, one commit

  • Cbor and CodecConfig::Cbor in cluster::codec, the crate optional in the cluster feature; rustdoc with the trade and the restart.
  • Unit tests: Postcard's four for Cbor, plus the rows above CBOR is for. The mismatch refusal already tests cbor as the other name (peer.rs 5343-5368).
  • Integration: tests/cluster.rs configures every node in test_config, so a TELLUS_REMOTING_CODEC variable runs the whole suite under CBOR, refs, replies, pub-sub and the legacy node included; a test-cluster-cbor recipe. A CI job for it, about 100 s, is your call.
  • By hand, reported in the PR: the suite and bench-cluster under both codecs, and the Compose demo once with codec: cbor through its faults.
  • Docs: the config table row, a CBOR paragraph in Rolling upgrades, and the Schema::VERSION rustdoc, which also states the bump rule.

Not in this PR

Mixed codecs or per-peer negotiation: you made the mismatch terminal in #262 as this codec's prerequisite, and a node speaking several codecs is a design of its own, which I'd plan separately if you want it. CBOR frames: frames carry no application type.

Two questions: which crate, and whether the docs relax "every change bumps VERSION" for CBOR or keep the bump as the recommendation with CBOR as the safety net.

Dominant language
Rust
Stars
37
Forks
1
Avg merge
6h 36m
Merged PRs (30d)
123

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tellus-rs/tellus

All issues in tellus-rs/tellus

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.