Study to compare t-Digest and REQ sketch

Open
#416 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
cpp
Domain
data, performance

Research direction

No files, tests, or benchmark entry point are named. Locate the t-Digest and REQ implementations and any existing performance harness, then compare HRA/LRA and default-scaling behavior across the input types described; done means a reproducible comparison with documented results.

Written by the indexing model from the issue text.

Description

Compare the performance of t-Digest with the closest competitor in the library, REQ sketch.
REQ sketch is the closest competitor because it prioritizes high rank accuracy (HRA mode) or low rank accuracy (LRA mode), unlike other quantile sketches (KLL, classic) with the same rank error for any rank.
There are a few obvious differences:

  • REQ sketch can work with any data type with a comparator, t-Digest is limited to numeric data (floating-point types)
  • REQ sketch retains and returns values observed in the input only - no notion of distance (only less than comparison), no interpolation. t-Digest is based on computing means and does interpolation.
  • t-Digest prioritizes both high rank and low rank accuracy at the same time with the default scaling function. Perhaps this can be changed with different scaling functions.
Dominant language
C++
Stars
273
Forks
88
Avg merge
1d 19h
Merged PRs (30d)
9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datasketches-cpp

All issues in apache/datasketches-cpp

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.