pingcap/tidb

Improve observability of fast reorg ADD INDEX progress

Open

#56,986 opened on Oct 30, 2024

View on GitHub
 (6 comments) (0 reactions) (1 assignee)Go (6,186 forks)batch import
component/ddlgood first issuetype/enhancement

Repository metrics

Stars
 (40,090 stars)
PR merge metrics
 (Avg merge 18d 2h) (292 merged PRs in 30d)

Description

Enhancement

currently we only let progress forward when KV is written to TiKV. When fast reorg is enabled, data will be batched on disk for a long time. No progress will be made during that stage and some users may suspect DDL is stuck.

So I propose we draw another line in metrics, which means "how many record KV we have read to assume the index KV".

Contributor guide

Research direction
Follow the existing DDL metrics pattern (e.g., in `metrics/ddl.go`) to add a new histogram or gauge for tracking the number of record KVs read during fast reorg ADD INDEX. Look at how the current progress metric is emitted and replicate the logic for the new metric.
Tech stack
go
Domain
backenddatabase
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoPrometheusTiDB internals (DDL)
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60