Compress TDigest can generate centroid with NaN mean

Closed
#702 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
java
Domain
data

Research direction

Start with the linked pull-request discussion and trace the TDigest compression and deserialization paths mentioned in the issue. Verify how centroid means and weights are validated, and confirm that stored centroids cannot contain NaN means or non-positive weights while rank/quantile behavior remains addressed separately.

Written by the indexing model from the issue text.

Description

See this thread https://github.com/apache/datasketches-rust/pull/23#discussion_r2622754228

Also we don't verify that the centroid mean is not NaN and the weight is not zero (should be positive even?) on deserialization. (Generally we can't assume the data to be deserialized is safe.)

I think the rank/quantile method can return NaN if the calculation result in that. But the stored centroid should not be NaN or else stable sort is unsound?

Dominant language
Java
Stars
958
Forks
226
Avg merge
3d 40m
Merged PRs (30d)
13

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datasketches-java

All issues in apache/datasketches-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.