commitment v3: a DB unwind does not restore a deleted record; other findings from the #24304 post-merge review
Maintainers usually reply within 1 day
@awskii is already working on this.
Since Nov 17, 2026.
Assessment
This issue has not been assessed yet.
Description
Findings from a review of #24304 as merged (fd679ca053). Point 1 is a defect for a node that runs with --experimental.commitment-v3: after a reorg it can reject a valid block. Points 2 to 6 are small. Legacy commitment is not affected by any of them.
1. A DB unwind does not restore a deleted v3 record
A reorg can bring back a stale commitment record. The next block that changes that storage is then rejected with invalid state root hash, or the fold fails with commitment v3: invalid node record.
Cause
DomainRoTx.unwind does not restore an empty previous value in the commitment domain (db/state/domain.go, from #26172):
if d.Name == kv.CommitmentDomain && len(diff.Value) == 0 {
continue
}
This is right for the legacy trie, where an empty previous value means that the branch was not in the trie. In v3 an empty value is the stored deletion of a node record, and reads depend on it.
The sequence:
- Key K has a record in an older step, in the DB or in files.
- A kept block deletes K at step S. The DB now holds an empty value for K at step S.
- A later block of step S writes K again. It replaces the same row, and its diff holds the empty previous value.
- That block is unwound. The row is deleted and the empty value is not written back, so the old record of K is the latest again.
The storage root record is read by its key, without the account leaf, so the next fold of that storage starts from the stale record. If nothing changes that storage again, the stale record stays as an orphan, and FoldCommitmentV3 reports it once it is in files.
#24304 got this with its merge of main on 2026-11-16. Each of the two changes is correct without the other.
How to reproduce
A unit test for db/state, package state. On main (29f1e11f97) it fails with Should be empty, but was [1]:
// A v3 record deletion is stored as an empty value. When a later block of the same step
// writes the record again and is unwound, the deletion must come back.
func TestDomain_UnwindRestoresDeletedCommitmentV3Record(t *testing.T) {
cfg := statecfg.Schema.CommitmentDomain
statecfg.EnableCommitmentV3Records(&cfg)
db, d := testDbAndDomainOfStep(t, cfg, 16, log.New())
tx, err := db.BeginRw(t.Context())
require.NoError(t, err)
defer tx.Rollback()
view := d.beginForTests()
defer view.Close()
writer := view.NewWriter(db)
defer writer.Close()
key, old := []byte("key"), []byte{1}
require.NoError(t, writer.PutWithPrev(key, old, 2, nil))
require.NoError(t, writer.DeleteWithPrev(key, 18, old))
writer.SetDiff(&kv.DomainDiff{})
require.NoError(t, writer.PutWithPrev(key, []byte{2}, 20, []byte{}))
diffs := writer.Diff().GetDiffSet()
require.NoError(t, writer.Flush(t.Context(), tx))
require.NoError(t, view.unwind(t.Context(), tx, 20, diffs, nil))
value, _, _, err := view.GetLatest(key, tx)
require.NoError(t, err)
require.Empty(t, value)
}
End to end through the exec module, with step size 64: a contract has slots 1 and 2. Block 25 clears both and block 26 sets slot 1. The fork choice moves to a side block 26' with a plain transfer, then side block 27' sets slot 2. With ERIGON_COMMITMENT_V3=true, 27' is rejected with invalid state root hash. With the flag off it is valid.
The end-to-end test, package execmoduletester
// Canonical: 24 transfers | clear slots 1 and 2 | set slot 1.
// Side: 24 transfers | clear slots 1 and 2 | transfer | set slot 2.
// With step size 64 the genesis storage is in step 0, and the clearing block and
// the block after it are in step 1.
func TestReorgAfterStorageEmptiedAndRefilled(t *testing.T) {
const stepSize, clearIdx = 64, 24
key, err := crypto.HexToECDSA("b71c71a67e1177ad4e901695e1b4b9ee17ae16c6668d313eac2f96dbcda3f291")
require.NoError(t, err)
sender := crypto.PubkeyToAddress(key.PublicKey)
store := common.HexToAddress("0xaa")
slot := func(i byte) (h common.Hash) { h[31] = i; return h }
m := New(t, WithStepSize(stepSize), WithKey(key), WithGenesisSpec(&types.Genesis{
Config: chain.AllProtocolChanges,
Alloc: types.GenesisAlloc{
sender: {Balance: new(big.Int).Exp(big.NewInt(10), big.NewInt(18), nil)},
// sstore(calldata[0:32], calldata[32:64])
store: {Code: []byte{0x60, 0x20, 0x35, 0x60, 0x00, 0x35, 0x55, 0x00}, Storage: map[common.Hash]common.Hash{slot(1): slot(1), slot(2): slot(1)}},
},
}))
signer := *types.LatestSignerForChainID(m.ChainConfig.ChainID)
gasPrice := uint256.NewInt(m.Genesis.BaseFee().Uint64() * 2)
sign := func(txn types.Transaction) types.Transaction {
signed, err := types.SignTx(txn, signer, key)
require.NoError(t, err)
return signed
}
sstore := func(nonce uint64, k, v byte) types.Transaction {
ks, vs := slot(k), slot(v)
return sign(types.NewTransaction(nonce, store, uint256.NewInt(0), 2_000_000, gasPrice, append(ks[:], vs[:]...)))
}
gen := func(side bool) func(int, *blockgen.BlockGen) {
return func(i int, b *blockgen.BlockGen) {
nonce := b.TxNonce(sender)
switch {
case i == clearIdx:
b.AddTx(sstore(nonce, 1, 0))
b.AddTx(sstore(nonce+1, 2, 0))
case i == clearIdx+1 && !side:
b.AddTx(sstore(nonce, 1, 7))
case i == clearIdx+2:
b.AddTx(sstore(nonce, 2, 9))
default:
b.AddTx(sign(types.NewTransaction(nonce, common.Address{1}, uint256.NewInt(1), 21_000, gasPrice, nil)))
}
}
}
canonical, err := m.GenerateChain(clearIdx+2, gen(false))
require.NoError(t, err)
side, err := m.GenerateChain(clearIdx+3, gen(true))
require.NoError(t, err)
ctx := t.Context()
require.NoError(t, m.InsertValidateAndUfc1By1(ctx, canonical.Blocks))
m.ExecModule.WaitIdle(ctx)
for _, block := range side.Blocks[clearIdx+1:] {
status, err := m.InsertBlocks(ctx, []*types.Block{block})
require.NoError(t, err)
require.Equal(t, execmodule.ExecutionStatusSuccess, status)
fcu, err := m.UpdateForkChoice(ctx, block.Header())
require.NoError(t, err)
require.Equalf(t, execmodule.ExecutionStatusSuccess, fcu.Status, "side block %d: %s", block.NumberU64(), fcu.ValidationError)
m.ExecModule.WaitIdle(ctx)
}
}
A random reorg test through the exec module (5 reorgs of depth 1 to 4 per seed, Prague and Amsterdam) fails for 8 of 40 seeds with v3 and for no seed with the flag off.
Fix
Adding && !d.CommitmentV3Records to the condition makes all of these pass, the random test included. TestDomain_UnwindDoesNotRestoreAbsentBranch then needs the legacy config: it takes statecfg.Schema.CommitmentDomain, which is the v3 config in the v3 CI leg, and its case is a legacy one.
Smaller points
db/state: a refusedconvert --v3prints the advice to add--no-history"if commitment history was pruned". When commitment history is complete but accounts, storage or code history does not start at step 0,rebuild --yesfails with[rebuild_commitment_history] root mismatch at block 0, afterrm-statehas already removed the commitment files.rebuild --yes --no-historythen works. The advice could name any pruned history.cmd/integration/commands:TestCommitmentRebuildWithoutHistoryAfterPrunedHistoryRefusalalso passes without--no-history, because its DB does not have the commitment-history flag set. So it does not show that the flag is needed.db/state:verifyCommitmentV3Historynow folds once per history file, as of the first commit. This sample reads entries that the earlier two samples per file did not, and misses others: with one byte changed in the account root record at a later txNum of a file, convert now accepts the file, and the earlier samples rejected it.cmd/utils: no test fails whenSetEthConfigstops callingsetCommitmentV3, so the node flag has no test.execution/commitment/backtesteralways builds a legacy trie. On a v3 datadir it stops inSeekCommitmentwith the hint to setERIGON_COMMITMENT_V3=true, also when it is set.integration commitment branchreads legacy keys, so on a v3 datadir it finds nothing and prints<empty>.
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.6k
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 820
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from erigontech/erigon
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
erigontech/erigon#26523 ·
Maintainers usually reply within 1 day
-
flaky test
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
erigontech/erigon#26230 ·
Maintainers usually reply within 1 day
-
txnprovider/txpool: pending nonce counts past gaps in the poolPossibly taken @yperbasis claimed this 7 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
erigontech/erigon#26014 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
tracing: expose custom error selector and raw payload in callTracer revertReasonPossibly taken @lupin012 claimed this 9 days ago. OpenRPC
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
erigontech/erigon#25928 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
txnprovider/txpool: remove redundant blob/commitment length checkPossibly taken @yperbasis claimed this 14 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
erigontech/erigon#25432 ·
Maintainers usually reply within 1 day
All issues in erigontech/erigon
Similar issues
-
enhancement exporter/awskinesis needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
open-telemetry/opentelemetry-collector-contrib#53334 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 83/100
SocialGouv/iterion#3376 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
seaweedfs/seaweedfs#12740 · 2 comments ·
Maintainers usually reply within 1 day
-
area:indexer bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
seanb4t/codegraph-go#385 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day