help wantedsig/executiontype/enhancement
Repository-Metriken
- Stars
- (40.090 Stars)
- PR-Merge-Metriken
- (Durchschn. Merge 14T 4h) (346 gemergte PRs in 30 T)
Beschreibung
Enhancement
Although we support spilling for unparallel HashAgg now, the memory usage of distinct is also out of control. The reason is that we need a IntSet/FloatSet/StringSet to check whether a value is distinct, and the Set maybe take up huge memory.
We need some spilling strategy for distinct function, to control the memory usage of SQL included distinct function.
Fortunately, only few aggregate function need to support distinct, so we only need consider the following distinct function: Count, Sum, Avg, GroupConcat
TiDB seems also support distinct for STDDEV_POP,STDDEV_SAMP, etc. But Mysql doesn't support the syntax