pingcap/tidb

Parallel evaluation for aggregate functions with distinct in hashAgg

Open

#17,075 opened on May 10, 2020

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Go (6,186 forks)batch import
help wantedsig/executiontype/enhancementtype/performance

Repository metrics

Stars
 (40,090 stars)
PR merge metrics
 (Avg merge 14d 4h) (346 merged PRs in 30d)

Description

Development Task

select count(distinct a) from t;

Currently if one of the aggregate functions of a SQL has a distinct evaluation, the hashAgg executor must run in a single thread. See https://github.com/pingcap/tidb/blob/3224393/executor/builder.go#L1207

We may find a method to evaluate it in parallel, maybe a sync.Map for deduplicating entries can be shared in multiple goroutines.

Contributor guide

Research direction
Study the current hashAgg implementation and design a parallel version using sync.Map or concurrent hash map. Analyze the distinct deduplication logic and propose a thread safe approach.
Tech stack
go
Domain
database
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoSQLconcurrency
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50