yugabyte/yugabyte-db

[DocDB] Handle tombstones correctly while decoding records using sst_dump

Open

#19.707 geöffnet am 27. Okt. 2023

Auf GitHub ansehen
 (2 Kommentare) (0 Reaktionen) (1 zugewiesene Person)C (1.003 Forks)batch import
area/docdbgood first issuekind/enhancementpriority/medium

Repository-Metriken

Stars
 (8.229 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 17T 21h) (92 gemergte PRs in 30 T)

Beschreibung

Jira Link: DB-8501

Description

It seems like we don't handle tombstones records correctly while decoding sst files using sst_dump, and instead output something like the below

SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
SubDocKey(DocKey(0x1210, [1], []), [ColumnId(11)]) [kStrongRead, kStrongWrite] HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/intent.cc:157): Expecting intent value to start with ValueType kTransactionId, but it is empty
TXN META 6995037d-2149-4a10-8cf5-c2cad08fdae8 -> Corruption (yb/util/uuid.cc:309): Invalid length of binary data with TransactionId '': 0 (expected 16)
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 1 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""
TXN REV 6995037d-2149-4a10-8cf5-c2cad08fdae8 HT{ physical: 1698422872751249 w: 2 } -> Corruption (yb/dockv/doc_key.cc:818): Error: failed decoding SubDocKey "": Found too few bytes in the end of a SubDocKey for a type-prefixed hybrid_time: ""

The above is definitely misleading when we inspect tablet data for debugging purposes.

Steps to repro-

  1. begin two transactions, perform a few updates.
  2. issue flush on one of the involved tablets using
yb-ts-cli --server_address=<> flush_tablet <tablet id>
  1. commit one of the transactions.
  2. issue a flush request again

The tablet directory would have 2 intents files, and when we try to decode the latest one, we would see corruption error like the above.

Few pointers from @spolitov sst_dump uses internal iterator, that fetches raw rocks db entries:

InternalIterator* iter =
      table_reader_->NewIterator(ReadOptions(verify_checksum_, false));
  uint64_t i = 0;
  if (has_from) {
    InternalKey ikey = InternalKey::MaxPossibleForUserKey(from_key);
    iter->Seek(ikey.Encode());
  } else {
    iter->SeekToFirst();
  }
  for (; iter->Valid(); iter->Next()) {

and it just ignores rocksdb value type.

FYI: @mdbr

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.

Contributor Guide