lance-format/lance

Column Statistic improvment list after MVP

Open

#5,857 opened on Jan 29, 2026

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Rust (695 forks)github user discovery
A-encodingA-formatenhancementfeaturehelp wanted

Repository metrics

Stars
 (6,582 stars)
PR merge metrics
 (Avg merge 6d 1h) (219 merged PRs in 30d)

Description

As we are wrapping up on the dataset level column statistics MVP, there are some high priority todo items: [ ] Support column stats generation for partial dataset. Currently the dataset will only be generated when all the files in the dataset has column stats written. A naive implementation is just to build the stats from the old fragment file rather than terminate early. [ ] Expose a Python API from the rust implementation. [ ] Exposing more stats in the discussion thread #4540

More could be found in the design doc

Contributor guide

Research direction
Study the existing column statistics implementation in the Rust codebase, understand the fragment manager and how stats are generated, then implement support for partial dataset by building stats from old fragments instead of terminating early. Also expose a Python API and add more statistics as per the discussion thread.
Tech stack
rust
Domain
database
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Rust
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30