[DocDB] Enable printing out why each RocksDB flush is being done · yugabyte/yugabyte-db#25615

(3 留言) (0 反應) (1 負責人)C (8,229 star) (1,003 fork)batch import

area/docdbgood first issuekind/enhancementpriority/medium

描述

Description

We have had two incidents now where inability to figure out why RocksDB flushes were occurring was key and extremely hard to figure out.

This task is to build instrumentation to make this easier in the future.

That is, create a runtime-settable gflag that turns on logging messages that for each flush describe why it is being done.

Reasons might include:

to stay under the global MemTable limit
to stay under the local MemTable limit
by request of a flush table RPC call
flush requested by DoCleanupIntentFiles

The above is almost certainly not a complete list. Ideally this task instruments all the calls to flush.

Although not ideal, it's probably acceptable for the reason log message to be given earlier than the log message announcing the flush itself as long as they can be correlated.

Ideally backport this to all versions in use.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

I confirm this issue does not contain any sensitive information.

貢獻者指南

技術棧: cppc
領域: databaseobservability
議題類型: feature
難度: 4
預計時間: 3-5 days
活動狀態: blocked
清晰度: clear
前置要求: C++ programmingRocksDB internalsYugabyteDB build systemgflags configuration
新手友善度: 20
研究方向: Identify all call sites where RocksDB flush is triggered (e.g., in src/rocksdb/db/db impl.cc). Add a runtime gflag (e.g., 'rocksdb flush reason logging') that enables logging the reason before each flush. The reason can be passed through a new argument to flush functions. Correlate with existing flush log messages. See comments on the issue for possible reasons list.