pingcap/tiflow

Filter out the range of index before the index is visible

Open

#634 opened on Jun 9, 2020

View on GitHubContributor guide
 (0 comments) (1 reaction) (0 assignees)Go (310 forks)auto 404
help wanted

Repository metrics

Stars
 (461 stars)
PR merge metrics
 (PR metrics pending)

Description

Feature Request

Is your feature request related to a problem? Please describe:

The process of add index is firstly writing index for existed rows and then make the index visible. For now, we subscribe to all regions in the range of table. Thus all operations for writing index for existed rows will be sent to ticdc from tikv as key/value operations, which are duplicated with add index in downstream.

Describe the feature you'd like:

If we do not subscribe to the range of an invisible index, then unnecessary data flow during adding an index can be avoided.

Contributor guide

Research direction
Study the TiCDC codebase to understand how index visibility and region subscription are handled. Look for the section where TiCDC subscribes to table ranges and identify how invisible indexes are filtered. The goal is to modify the subscription logic to exclude ranges of invisible indexes during the 'add index' process.
Tech stack
go
Domain
backenddatabasedata
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoTiDB basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40