`lwk_boltz`/`lwk_common`: store entry keys moved from a `sha256(key)[..12]` nonce to the tagged `DeterministicNonceHash` nonce, changing the on-disk address of every stored key with no migration

Start with lwk_common/src/encrypt.rs and the key encoding and lookup paths in lwk_common/src/store.rs; compare the deterministic nonce derivation with the parent revision's sha256(plaintext)[..12] behavior. Check existing store compatibility tests and add coverage showing that a key written with the previous derivation remains readable. Done means the compatibility behavior is tested and existing entries can still be found.

Summary

The deterministic entry-key derivation changed: encrypt_with_deterministic_nonce in lwk_common/src/encrypt.rs, called from the key path at lwk_common/src/store.rs:479, now derives its AEAD nonce from DeterministicNonceHash rather than sha256(plaintext)[..12], so the same plaintext key name is stored under a different address. Only entry keys are affected; the value path at store.rs:511 is unchanged. Observable consequence: a store written by the previous revision is no longer readable by this revision.

Environment

• Repository: Blockstream/lwk
• Current default branch: commit f2c64c74d5f9ef1079e842798a323d82e7dfaa89
• Reproduced at commit f771d1dc863d7db7221088426d50613c9e3828f0; still present in the current code at f2c64c74d5f9ef1079e842798a323d82e7dfaa89, at lwk_common/src/encrypt.rs:115-124 (the helper, which was lwk_common/src/crypto.rs:96 at the analysed commit), used at lwk_common/src/store.rs:479, with the value path at lwk_common/src/store.rs:511.
• Toolchain: Rust 1.85.1 (RUSTUP_TOOLCHAIN=1.85.1-aarch64-unknown-linux-gnu); the executed run used a container with the network disabled (--network=none) and cargo --offline.

The parent revision's key nonce, for comparison, was sha256(plaintext)[..12] (see the observed output below).

Expected behavior

The stored representation of a plaintext key is part of the on-disk format. A revision that changes how entry keys are derived should either keep the previous derivation — so the same plaintext key keeps the same stored address — or keep both representations readable and re-encrypt an entry when it is found under the old one. If the new derivation must be a hard cut, the change should at least be visible as a format-identity change rather than silently applied to every stored key.

Actual behavior

Observed output of the run described above:

TARGET f771d1dc:
parent nonce (sha256(key)[..12])            : e146c4f7cd4a2a6b240c6d8f
target nonce (DeterministicNonceHash[..12]) : ebfa0a7439b9c223daccc4ed
nonces equal                                : false
=== EXPERIMENT: same-revision round trip ===
store key this revision writes (from its own encrypt_key): 5686bfb1a65456f90adb9735206dbdc3eef430160e232a7fb9163f3e149bbeedd4db6f
written       : ["swap1", "swap2"]
read back     : ["swap1", "swap2"]
=== EXPERIMENT: value channel ===
ciphertext len / plaintext len+12+16 : 41 / 41
roundtrip ok                         : true

PARENT a87d891 (control):
store key this revision writes (from its own encrypt_key): b626c038e65c56398a305db0b690de74716feb63200a8a4ee9776c8096dcee208de9fb
written       : ["swap1", "swap2"]
read back     : ["swap1", "swap2"]
=== EXPERIMENT: value channel ===
ciphertext len / plaintext len+12+16 : 41 / 41
roundtrip ok                         : true

For the identical cipher and the identical plaintext key, the two revisions write different store keys (5686bfb1… vs b626c038…). The only differing input is the 12-byte nonce (ebfa0a74… vs e146c4f7…), and because that nonce is deterministic the difference is reproducible rather than accidental. The value channel is unchanged: the random 12-byte prefix plus ciphertext layout gives the same length in both builds (41 = 13 + 12 + 16) and the round trip succeeds, so nothing about values changed identity — only keys did. The same-revision round trip confirms the new scheme is internally consistent, i.e. the break only appears when data written by the previous revision is read.

Reproduced with:

git clone https://github.com/Blockstream/lwk.git && cd lwk
git checkout f771d1dc863d7db7221088426d50613c9e3828f0
# in a scratch copy, add a #[cfg(test)] test that calls EncryptedStore::effective_key
cargo test -p lwk_common --lib -- --nocapture
git checkout a87d89122a0631715602908d5be7ed9643f530a6
cargo test -p lwk_common --lib -- --nocapture

store key written under f771d1dc: 5686bfb1a65456f90adb9735206dbdc3eef430160e232a7fb9163f3e149bbeedd4db6f
store key written under a87d89122: b626c038e65c56398a305db0b690de74716feb63200a8a4ee9776c8096dcee208de9fb

Current code at the default branch (verbatim). The deterministic nonce is still the tagged hash:

// lwk_common/src/encrypt.rs:115-119
pub fn encrypt_with_deterministic_nonce(
 cipher: &mut Aes256GcmSiv,
 plaintext: &[u8],
) -> Result<Vec<u8>, EncryptError> {
 let hash = DeterministicNonceHash::hash(plaintext);

That deterministic output is hex-encoded to form the stored key, while the value path still uses a random nonce:

// lwk_common/src/store.rs:479-481
 let encrypted = encrypt_with_deterministic_nonce(&mut cipher, key.as_ref())
 .map_err(EncryptedStoreError::Encrypt)?;
 Ok(encrypted.to_lower_hex_string().into_bytes())

// lwk_common/src/store.rs:511
 let ciphertext = encrypt_with_random_nonce(&mut cipher, value.as_ref())

Root cause

The commit routed the Boltz store's key encryption through the new shared helper, and that helper derives its nonce from DeterministicNonceHash — a tagged hash (LWK-Deterministic-Nonce/1.0) — instead of the previous sha256(plaintext)[..12]. Three properties make that a representation change rather than a refactor:

1. The stored entry key is deterministic: it is the hex encoding of encrypt_with_deterministic_nonce(&mut cipher, key.as_ref()) (lwk_common/src/store.rs:479-481), and the nonce is a pure function of the plaintext key name, so for a fixed AES key the whole store key is a pure function of the key name.
2. That pure function changed. sha256(plaintext)[..12] and DeterministicNonceHash::hash(plaintext)[..12] are different 12-byte values for the same input (e146c4f7… vs ebfa0a74… in the run above), so every plaintext key now has a different stored address.
3. Nothing reads the old address. The helper only ever derives the tagged-hash nonce, and there is no migration, dual-read or re-encryption path over the previously written representation.

The asymmetry is explicit in the current code: keys go through the deterministic nonce (lwk_common/src/store.rs:479) while values go through a random one (:511), so exactly one half of the store changed identity. The commit under test was a broader change (the helper was extracted and the value-side code was refactored behaviour-neutrally), which is why the value channel is byte-compatible and only the key derivation is not.

The defect is live on the current default branch: the deterministic nonce is still DeterministicNonceHash::hash(plaintext) (lwk_common/src/encrypt.rs:115-119), the key is still hex-encoded from that helper's output (lwk_common/src/store.rs:479-481), and the value path is still the random-nonce one (:511). The module was later renamed from lwk_common/src/crypto.rs to lwk_common/src/encrypt.rs and the Boltz-specific encrypt_key/encrypt_value now live on EncryptedStore in lwk_common/src/store.rs, but the derivation itself is unchanged, so the rename and refactor do not repair the representation break.

Suggested fix

Treat the nonce derivation as part of the on-disk format identity. Two options:

• Keep the previous derivation for entry keys, i.e. sha256(plaintext)[..12], so existing records keep their address. If the tagged hash was introduced only for consistency across the shared helper, this is the smallest change.
• Read both representations. Derive the store key under the tagged-hash nonce, and on a miss also derive it under the legacy sha256(plaintext)[..12] nonce; if that entry exists, decrypt it and rewrite it under the new derivation (re-encryption is cheap because the plaintext key name is known at the call site).

Either way, add a regression test that writes an entry through the previous derivation and asserts it is still readable after the change — the same style of test the project already uses to pin store-key compatibility elsewhere. If the project decides the tagged derivation must stay as-is because no released version ever wrote the old address, then that decision is worth stating explicitly in the change, because the break is otherwise silent.