yugabyte/yugabyte-db

[DocDB] Handle tombstones correctly while decoding records using sst_dump

Open

#19,707 建立於 2023年10月27日

在 GitHub 查看
 (2 留言) (0 反應) (1 負責人)C (1,003 fork)batch import
area/docdbgood first issuekind/enhancementpriority/medium

倉庫指標

Star
 (8,229 star)
PR 合併指標
 (平均合併 17天 21小時) (30 天內合併 92 個 PR)

描述

Jira Link: DB-8501

Description

It seems like we don't handle tombstones records correctly while decoding sst files using sst_dump, and instead output something like the below

SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), [ColumnId(11)]) [kStrongRead, kStrongWrite] HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
TXN META 6995037d-2149-4a10-8cf5-c2cad08fdae8 -> Corruption (yb/util/uuid.cc:309): Invalid length of binary data with TransactionId '': 0 (expected 16)
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""

The above is definitely misleading when we inspect tablet data for debugging purposes.

Steps to repro-

  1. begin two transactions, perform a few updates.
  2. issue flush on one of the involved tablets using
yb-ts-cli --server_address=<> flush_tablet <tablet id>
  1. commit one of the transactions.
  2. issue a flush request again

The tablet directory would have 2 intents files, and when we try to decode the latest one, we would see corruption error like the above.

Few pointers from @spolitov sst_dump uses internal iterator, that fetches raw rocks db entries:

InternalIterator* iter =
      table_reader_->NewIterator(ReadOptions(verify_checksum_, false));
  uint64_t i = 0;
  if (has_from) {
    InternalKey ikey = InternalKey::MaxPossibleForUserKey(from_key);
    iter->Seek(ikey.Encode());
  } else {
    iter->SeekToFirst();
  }
  for (; iter->Valid(); iter->Next()) {

and it just ignores rocksdb value type.

FYI: @mdbr

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.

貢獻者指南