pingcap/tidb

Could drainer to kafka support multi partition

Open

#17,525 opened on May 29, 2020

View on GitHub
 (0 comments) (0 reactions) (1 assignee)Go (6,186 forks)batch import
feature/acceptedhelp wantedtype/feature-request

Repository metrics

Stars
 (40,090 stars)
PR merge metrics
 (Avg merge 14d 4h) (346 merged PRs in 30d)

Description

Description

Could drainer to Kafka support multi-partition. Reported from https://asktug.com/t/topic/34614/27

Category

Performance

Value

Support multi-partition could improve Kafka performance

Value Score(Optional): 3

Workload Estimation(Optional)

N/A

Contributor guide

Research direction
Investigate the drainer Kafka producer code and implement partition key assignment based on message key or hash.
Tech stack
go
Domain
backend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoKafkaTiDB
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30