Validator does not distinguish bignums (tags 2/3) from plain ints

Read Gen.hs, validateInteger's validateBigInt, and the cborg decoder primitives first; then trace the custom-validator API and generator handling of VBignum literals. Check how Term producers and consumers represent wire bignums, including indefinite payloads and byte controls. Done means representation-sensitive validation works without losing sign or payload information, and the stated generator and API conventions are addressed.

Summary

The validator decodes input with cborg's decodeTerm, which folds tags 2/3 into TInteger at the token level (cborg maps wire headers 0xc2/0xc3 to TypeInteger and decodeInteger consumes the tag + payload). By the time validateTerm runs, the distinction between "major type 0/1" and "tag 2/3 bignum" is erased — but CDDL's matching rules are representation-sensitive: per RFC 8610 (§3.3, Appendix C/D), uint = #0, nint = #1, and numeric literals match only major type 0/1 data items, never tag 2/3 items. That's why the prelude needs integer = int / bigint and unsigned = uint / biguint.

This matters for interop: encoders other than cborg may legally emit valid-but-non-preferred forms (RFC 8949 §3.4.3 — bignums with leading zeroes MUST be accepted; values that fit major type 0/1 wrapped in tag 2/3 are valid; indefinite-length payloads are valid outside deterministic encoding).

Observed behavior

Hard failure on valid CBOR:

• a = biguint vs C2 5F 41 05 FF (tag 2, indefinite-length payload) → DecodingFailed "expected integer". Valid CBOR per RFC 8949 §3.4.3/§2 (definite vs. indefinite is a serialization variant; only deterministic encoding §4.2.1 forbids it). Known issue — see the TODO at Gen.hs (disableTwiddle for tags 2/3).

Wrong results (sign not checked — validateInteger treats Tag 2 and Tag 3 rules identically):

• a = biguint vs C3 49 010000000000000000 (negative bignum) → accepted
• a = bignint vs C2 49 010000000000000000 (positive bignum) → accepted
• a = biguint vs 20 (plain −1) → accepted

Too strict (payload bytes are gone, so byte controls run against mempty — see TODO in validateInteger's validateBigInt):

• a = #6.2(bstr .size 9) vs C2 49 010000000000000000 (exactly 9 payload bytes) → rejected

Too lenient (spec deviation, fails open):

• a = {1: uint} vs map key encoded as C2 41 01 → accepted; Appendix C: a literal "matches only a data item with that specific value", and a tag-2 item is a different data-model value

What already works correctly: bignums ≥ 2^64 accepted for biguint/bigint/integer and rejected for int/uint; leading-zero bignum payloads accepted by value; non-minimal int widths accepted (correct — CDDL matches at the data-model level); generator output is preferred serialization and round-trips.

Root cause and proposed fix

The information loss is unrecoverable at the Term level in general: TInteger i with |i| < 2^63 can only come from a bignum, but values in [2^63, 2^64) (and the negative mirror) decode to TInteger from both major type 0/1 and tags 2/3. So a TInteger-only fix in the validator cannot be complete, and can't fix the decode failure or byte controls at all.

Proposed: a cuddle-local variant of cborg's decodeTerm that keeps wire bignums as TTagged 2/3 (TBytes …) instead of folding them. This is straightforward because in cborg's decoder primitives, peekTokenType returns TypeInteger exclusively for wire headers 0xc2/0xc3 (genuine ints come back as TypeUInt/TypeUInt64/TypeNInt/TypeNInt64), and decodeTag happily consumes 0xc2/0xc3 as ordinary tags. So the TypeInteger branch can do decodeTag + payload decode instead of decodeInteger. Verified this preserves indefinite-length payloads too (C2 5F 41 05 FF → TTagged 2 (TBytesI …)).

This fixes all four gaps: representation-correct matching for uint/int/literals, sign correctness for biguint/bignint, working .size/.cbor controls on #6.2(bstr …) payloads, and no decode failure on indefinite-length payloads.

Note this establishes a convention on Term that differs from cborg's: TInt/TInteger would mean "major type 0/1 on the wire" and TTagged 2/3 (TBytes …) "bignum on the wire" (cborg's encodeTerm already round-trips this faithfully). The convention must then hold for all Term producers, not just the decoder:

• the generator currently emits TInteger for VBignum literals — it should emit TTagged 2/3 (TBytes minimalBytes);
• the custom-validator API wraps every integer as SingleTerm (TInteger i) and accepts user-constructed terms via validateFromName/validateAgainst — the convention needs documenting at that API boundary (or, more heavyweight, a provenance newtype / dedicated term type, in the spirit of CanonicalTerm).

Value-level equivalence (prelude integer/unsigned, toCanonical map-key dedup, matching literals > 2^64−1 against bignums by numeric value) then becomes an explicit conversion where intended. On that last point: RFC 8610 never actually defines what a literal ≥ 2^64 matches (the basic data model has no such integer) — matching it against tag 2/3 by value is a pragmatic extension, same as the reference cddl tool, and worth stating in the docs.

Spec references

• RFC 8949 §3.4.3 (bignums: content must be a byte string; leading zeroes MUST be accepted; preferred serialization), §4.2.1/§4.2.2 (deterministic encoding), §5.3.2 (tag validity)
• RFC 8610 §2.2.3 (representation types are data-model level), §3.3 (uint/nint), Appendix C (type2 = value: "matches only a data item with that specific value (no conversions defined)"), Appendix D (prelude: uint = #0, biguint = #6.2(bstr), unsigned = uint / biguint, …)