categorical histogram: the unique bar re-counts singletons already shown as top values
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 85/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- data-visualization
Research direction
Start in histogram.py at categorical_dict and follow how categorical_histogram separates the top values from the remainder; the issue includes a reproducer using pandas and notes that the shared function also serves polars. Verify that the unique count includes only singleton values outside the top values, and that the displayed bars no longer exceed 100% for the reproducer.
Written by the indexing model from the issue text.
Description
Problem
categorical_dict (histogram.py:78) sets unique_count = sum(val_counts == 1) over every value, including the top 7 that categorical_histogram already draws one by one. When fewer than 7 values repeat, the singletons in the top 7 are counted twice and the bars add up to more than 100%. Sixteen rows with counts 5, 3 and eight singletons draw a 31, b 19, five singletons at 6 each, and unique 50, 130% in total. A one-row column reads 1.5 100 and unique 100. Pandas and polars share the function. #365 reported unique at 100% on a polars bool column.
import pandas as pd
from buckaroo.customizations.pd_stats_v2 import PD_ANALYSIS_V2
from buckaroo.pluggable_analysis_framework.stat_pipeline import StatPipeline
sd, _ = StatPipeline(PD_ANALYSIS_V2, unit_test=False).process_df(pd.DataFrame({"a": ["a"] * 5 + ["b"] * 3 + [f"s{i}" for i in range(8)]}))
print([(b["name"], b.get("cat_pop", b.get("unique"))) for b in sd["a"]["histogram"]])
# [('a', 31.0), ('b', 19.0), ('s0', 6.0), ('s1', 6.0), ('s2', 6.0), ('s3', 6.0), ('s4', 6.0), ('unique', 50.0)]
Suggested fix
Count only the singletons outside the top values (rest_vals == 1).
Context
Found while probing histogram edge cases after #1069. Reproduces on main at e88ace91.
- Dominant language
- Python
- Stars
- 685
- Forks
- 17
- Avg merge
- 23h 34m
- Merged PRs (30d)
- 24
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from buckaroo-data/buckaroo
-
Histograms JS
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
buckaroo-data/buckaroo#1089 · 1 comment ·
Maintainers usually reply within 1 day
-
Histograms
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
buckaroo-data/buckaroo#1087 · 1 comment ·
Maintainers usually reply within 1 day
-
Histograms
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
buckaroo-data/buckaroo#1086 · 1 comment ·
Maintainers usually reply within 1 day
-
Histograms
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
buckaroo-data/buckaroo#1082 · 1 comment ·
Maintainers usually reply within 1 day
-
histogram: numeric bars are a percentage of the bucketed values, the NA bar a percentage of all rowsOpenHistograms
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
buckaroo-data/buckaroo#1079 · 1 comment ·
Maintainers usually reply within 1 day
All issues in buckaroo-data/buckaroo
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
nansencenter/geodataset#39 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
awslabs/visual-asset-management-system#804 ·
Maintainers usually reply within 1 day
-
follow-up
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
odoo/odoo#297927 · 1 comment ·
Maintainers usually reply within 1 day
-
link-rot
Difficulty 1/5 Under an hour Newbie friendliness 67/100