yugabyte/yugabyte-db
GitHub で見る[DocDB] Handle tombstones correctly while decoding records using sst_dump
Open
#19,707 opened on 2023年10月27日
area/docdbgood first issuekind/enhancementpriority/medium
Repository metrics
- Stars
- (8,229 stars)
- PR merge metrics
- (平均マージ 17d 21h) (30d で 92 merged PRs)
説明
Jira Link: DB-8501
Description
It seems like we don't handle tombstones records correctly while decoding sst files using sst_dump, and instead output something like the below
SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), [ColumnId(11)]) [kStrongRead, kStrongWrite] HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
TXN META 6995037d-2149-4a10-8cf5-c2cad08fdae8 -> Corruption (yb/util/uuid.cc:309): Invalid length of binary data with TransactionId '': 0 (expected 16)
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
The above is definitely misleading when we inspect tablet data for debugging purposes.
Steps to repro-
- begin two transactions, perform a few updates.
- issue flush on one of the involved tablets using
yb-ts-cli --server_address=<> flush_tablet <tablet id>
- commit one of the transactions.
- issue a flush request again
The tablet directory would have 2 intents files, and when we try to decode the latest one, we would see corruption error like the above.
Few pointers from @spolitov sst_dump uses internal iterator, that fetches raw rocks db entries:
InternalIterator* iter =
table_reader_->NewIterator(ReadOptions(verify_checksum_, false));
uint64_t i = 0;
if (has_from) {
InternalKey ikey = InternalKey::MaxPossibleForUserKey(from_key);
iter->Seek(ikey.Encode());
} else {
iter->SeekToFirst();
}
for (; iter->Valid(); iter->Next()) {
and it just ignores rocksdb value type.
FYI: @mdbr
Issue Type
kind/enhancement
Warning: Please confirm that this issue does not contain any sensitive information
- I confirm this issue does not contain any sensitive information.