pingcap/tidb

Add SQL diagnosis rule: threshold check for tikv_append_log_duration

Open

#17,941 opened on Jun 11, 2020

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (6,186 forks)batch import
good first issuehelp wantedtype/enhancement

Repository metrics

Stars
 (40,090 stars)
PR merge metrics
 (Avg merge 14d 4h) (346 merged PRs in 30d)

Description

Development Task

When Raft append log duration, P99 Value > 500ms, then append an warning in INSPECTION_RESULT table, this may be caused by:

  • write disk too slow, check the disk latency.
  • It may trigger the RocksDB Write Stall.

Refer

  • You can query the tikv_append_log_duration table to get metrics_schema.tikv_append_log_duration value. Here is the related doc.
  • You can refer the thresholdCheckInspection implementation in TiDB.

If you have any questions or want to discuss something, you can chat in the sig-dashboard slack channel and @ chenshuang.

Contributor guide

Research direction
Inspect the existing `thresholdCheckInspection` implementation in TiDB and understand the metrics schema table `tikv append log duration`. Then add a new rule to check if P99 > 500ms and insert a warning in `INSPECTION RESULT`.
Tech stack
go
Domain
databaseobservability
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoSQL
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65