pingcap/tiflow

Optimization for DDL execution

Open

#866 opened on Aug 20, 2020

View on GitHubContributor guide
 (1 comment) (0 reactions) (1 assignee)Go (310 forks)auto 404
component/replica-modelcomponent/sinkhelp wanted

Repository metrics

Stars
 (461 stars)
PR merge metrics
 (PR metrics pending)

Description

Feature Request

Is your feature request related to a problem? Please describe:

In the current replication model, DDL is executed synchronously and blocks all DMLs in other tables. This could cause replication block in some scenarios

  • the DDL is slow, such add index to a large table.
  • the replication of one table is slow, other tables will have to wait for this table catch up because the DDL is a strong barrier.

Describe the feature you'd like:

  • Currently DDL execution has no timeout, should we add some restriction. https://github.com/pingcap/ticdc/pull/936
  • Once the add index command has been sent to TiDB, even the client is timeout, we can ignore this error and continue replication.
  • Could we add some mechanism to allow single table DDL doesn't block DML replication of other tables.

Contributor guide

Research direction
Investigate the current DDL execution in TiCDC replication, understand how it blocks DMLs, and propose a timeout mechanism or non blocking approach for single table DDLs to allow other tables to continue replication.
Tech stack
go
Domain
backend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GitGo
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40