pingcap/tidb

Support spilling for distinct aggregate function

Open

#27,092 建立於 2021年8月11日

在 GitHub 查看
 (4 留言) (0 反應) (1 負責人)Go (6,186 fork)batch import
help wantedsig/executiontype/enhancement

倉庫指標

Star
 (40,090 star)
PR 合併指標
 (平均合併 14天 4小時) (30 天內合併 346 個 PR)

描述

Enhancement

Although we support spilling for unparallel HashAgg now, the memory usage of distinct is also out of control. The reason is that we need a IntSet/FloatSet/StringSet to check whether a value is distinct, and the Set maybe take up huge memory.

We need some spilling strategy for distinct function, to control the memory usage of SQL included distinct function.

Fortunately, only few aggregate function need to support distinct, so we only need consider the following distinct function: Count, Sum, Avg, GroupConcat

TiDB seems also support distinct for STDDEV_POP,STDDEV_SAMP, etc. But Mysql doesn't support the syntax

貢獻者指南