can we use stable sort for tdigest
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 58/100
Research direction
Start in the t-digest merge logic where centroids are sorted by value and index. Compare that ordering with core/src/main/java/com/tdunning/math/stats/Sort.java referenced in the issue, then verify that equal-value centroids are ordered deterministically and produce consistent merge results.
Written by the indexing model from the issue text.
Description
the sort on centroids (v,w) -> (value, weight) in tdigest merge uses the sort based on value (v) and index -> therefore if we have centroids like (200,1)(100,3)(300,2)(100,1) then at merge it can be sorted to 2 states :
- (100,1)(100,3)(200,1)(300,2)
- (100,3)(100,1)(200,1)(300,2)
due to which the merging result can differ
other than that we can use sorting based on value (v) and weight (w) as in
https://github.com/tdunning/t-digest/blob/main/core/src/main/java/com/tdunning/math/stats/Sort.java
and this will make the sort stable and remove variance in result
- Dominant language
- Java
- Stars
- 958
- Forks
- 226
- Avg merge
- 3d 40m
- Merged PRs (30d)
- 13
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datasketches-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
apache/datasketches-java#731 · 16 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
apache/datasketches-java#720 · 3 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
apache/datasketches-java#693 · 13 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
apache/datasketches-java#647 · 5 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
apache/datasketches-java#569 ·
All issues in apache/datasketches-java
Similar issues
-
bug untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
opensearch-project/ml-commons#5094 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
emitter:client:csharp feature
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
affects/8.10 affects/8.9 component/clients kind/bug likelihood/mid severity/mid
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Two open-case totals on one screen: the Programs tile says 15,858 and the nav badge says 15,868 Openbug frontend maui-pilot
Difficulty 2/5 1-3 hours Newbie friendliness 72/100