Public API accepts non-canonical expanded ML-KEM decapsulation key representations and exposes inconsistent handling of the resulting state

Start by reading decapsulation_key.rs at from_expanded(), PartialEq, and KeyExport::to_bytes(), then compare the validation path in pke.rs with the error handling in pkcs8.rs. Reproduce the non-canonical expanded-key and export cases, add regression coverage, and make the chosen validation and failure behavior consistent with the project’s intended API contract.

Public API accepts non-canonical expanded ML-KEM decapsulation key representations and exposes inconsistent handling of the resulting state

Summary

DecapsulationKey::from_expanded() (public, #[deprecated]) and its trait
wrapper ExpandedKeyEncoding::from_expanded_bytes() (line 272, delegates at
line 273) accept non-canonical 12-bit coefficient encodings in the decryption
key region without validation. Both are public APIs — from_expanded_bytes()
delegates directly to from_expanded(), which in turn calls the crate-internal
DecryptionKey::from_bytes() (infallible, no modulus check).

The accepted representation produces an internal state (d = None, line 77)
that is observably different from seed-constructed keys in one specific
capability (to_seed), despite PartialEq reporting equality (comment at
line 131, implementation at line 137 — intentionally omits d). This state
divergence propagates to KeyExport::to_bytes() as a panic (line 232), while
the same condition is handled gracefully as an error in the PKCS#8 path
(line 162).

No cryptographic divergence was observed: 546 non-canonical representations
across ML-KEM-512, ML-KEM-768, and ML-KEM-1024 all produced identical shared
secrets compared to their canonical equivalents.

All line references are against commit 1d535a1178476a55b308efb510f86bdf7fd5330b.

Tested revision

Commit: 1d535a1178476a55b308efb510f86bdf7fd5330b

Environment

• rustc 1.96.0 (ac68faa20 2026-05-25)
• cargo 1.96.0 (30a34c682 2026-05-25)
• Linux 6.17.0-40-generic x86_64

Observed properties

1. Non-canonical dk acceptance (representation)

The public entry points are DecapsulationKey::from_expanded() (line 64,
pub, #[deprecated]) and ExpandedKeyEncoding::from_expanded_bytes()
(line 272, pub trait method, delegates to from_expanded() at line 273).
Both are compilable — the deprecation notice is the only signal to consumers.

from_expanded() calls the crate-internal DecryptionKey::from_bytes() in
pke.rs (line 111). This function is infallible — it calls decode_u12,
which applies byte_decode with a 12-bit mask (& 0x0FFF) followed by
small_reduce (conditional subtraction of Q=3329). Coefficients in
[3329, 4095] are silently reduced to [0, 766].

This contrasts with EncryptionKey::from_bytes() (line 168), which performs
the FIPS 203 §7.2 modulus check via encode-decode round-trip and returns
InvalidKey on mismatch.

Note: from_expanded() does validate the encryption key portion of the
expanded representation (via EncryptionKey::from_bytes()?) and checks the
hash (ek.h() != *h). The decryption key portion is the one accepted without
validation.

Reproduction: Modify any dk_pke coefficient c < 767 to c + 3329 in an
expanded key. from_expanded_bytes() accepts it. Re-export via
to_expanded_bytes() produces the original canonical form.

Falsification: 195/195 non-canonical coefficients accepted (ML-KEM-768,
single key). 546/546 non-canonical representations across all three parameter
sets produced identical shared secrets.

2. Panic in KeyExport::to_bytes() (availability)

from_expanded() sets d = None (line 77 of decapsulation_key.rs).
KeyExport::to_bytes() calls
self.to_seed().expect("should be initialized from a seed") (line 232),
which panics.

The same condition is handled in pkcs8.rs (line 162) via
self.to_seed().ok_or(pkcs8::KeyError::Invalid)?.

Reproduction:

#[allow(deprecated)]
let dk = DecapsulationKey::from_expanded_bytes(&expanded).unwrap();
let _seed: Seed = dk.to_bytes(); // panics at decapsulation_key.rs:232:24

3. PartialEq hides capability divergence (API contract)

The PartialEq implementation intentionally omits d from comparison
(comment at line 131, implementation at line 137 of decapsulation_key.rs).
Two keys that compare equal may have different capabilities:

Operation	d = Some (from_seed)	d = None (from_expanded)
decapsulate	works	works (identical result)
encapsulation_key	works	works (identical result)
to_expanded_bytes	works	works (identical result)
to_seed()	Some(seed)	None
KeyExport::to_bytes()	returns seed	panics
PKCS#8 to_pkcs8_der()	works	returns KeyError::Invalid

Security characterization

No cryptographic divergence was observed. The demonstrated impact is:

1. A public-API panic (denial of service) reachable through the deprecated but
   compilable from_expanded path followed by KeyExport::to_bytes().
2. An inconsistent error/panic contract for the same internal state across
   KeyExport vs PKCS#8.
3. An equality contract that does not reflect functional equivalence.

Normative note

FIPS 203 §7.2 specifies a modulus check for encapsulation keys (implemented).
The corresponding validation requirements for expanded decapsulation key
representations under §7.3 remain under review by the reporter. This issue
does not assert a FIPS 203 conformance gap — it reports the observed behavior
and its consequences.

Suggested remediation

1. Replace .expect() in KeyExport::to_bytes() (line 232) with error
   handling consistent with the PKCS#8 path (e.g., return Option<Seed>
   or propagate an error).
2. Consider adding a modulus check to DecryptionKey::from_bytes() analogous
   to the EncryptionKey round-trip validation, or document the asymmetry
   as an intentional design decision.
3. Consider documenting the PartialEq semantics regarding d or providing
   an fn is_seed_exportable(&self) -> bool method.