Quantile Support for Pre-Aggregations

Is your feature request related to a problem? Please describe. Much of our aggregations are based on quantiles (e.g. 0.95 -> 95p). It is well-known that you cannot merge quantiles without having all of the original data, which prevents such aggregations in pre-aggregations. It would be very nice to have pre-computed quantiles that can be merged, similar to counts, sums, etc.

First, does the community care about quantile-based pre-aggregations? If the answer is no, then we can just close this out.

If there seems to be a use for this, I'd be happy to help.

I mentioned this at the community meeting on 06/09/2021 (@rpaik).

Describe the solution you'd like

References:

• T-digest paper: https://arxiv.org/pdf/1902.04023.pdf
• T-digest reference implementation (there is a JS port): https://github.com/tdunning/t-digest

There are digest-based data structures, such as t-digests, that can produce accurate, bounded-size, mergeable quantile approximations. They are typically used in streaming or map-reduce-like settings for efficiently/concurrently combining rank statistics, but I think they would also be useful in pre-aggregations. There is a definite tradeoff between space and accuracy, so tuning could be tricky, but it seems that there are decent defaults that would suit most use cases.