pingcap/tidb

Improve the performance of `Sort` by parallel

Open

#14,417 opened on 2020年1月9日

GitHub で見る
 (1 comment) (0 reactions) (0 assignees)Go (40,090 stars) (6,186 forks)batch import
help wantedsig/executiontype/enhancement

説明

Feature Request

Is your feature request related to a problem? Please describe:

Now the Sort executor is executed serially. We can run it parallelly to speed up.

Describe the feature you'd like:

Considering using a parallel framework, to run each sort partition in parallel. And then using a merge sort to generate the final result. We can refer to this proposal https://github.com/pingcap/tidb/pull/14238#issuecomment-569880893 and reuse it (after this pr merge).

Describe alternatives you've considered:

Task:

  1. Implement parallel sort.
  • Implement merge sort algorithm above Shuffle operator.
  • Implement plan builder for parallel sort.
  1. Cost Model, explain analyze information.
  • Change the cost model of sort if it will run parallelly.
  • Change the explain analyze result for parallel sort.

Teachability, Documentation, Adoption, Migration Strategy:

コントリビューターガイド

技術スタック
go
領域
backenddatabaseperformance
Issue 種別
feature
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
5
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
over 1 week
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
clear
前提条件
GoTiDB internalsMerge sort algorithmParallel execution
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
10
調査方針
Review the referenced proposal in PR #14238 and the existing sort executor code in TiDB. Focus on integrating a merge sort above the Shuffle operator and modifying the plan builder. Understand the current cost model and explain analyze output to plan changes for parallel sort.
Improve the performance of `Sort` by parallel · pingcap/tidb#14417 | Good First Issue