[Feature] Support Mosaic file format
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start by tracing the Paimon C++ file-format SPI and comparing its current schema, reader, writer, and statistics handling with Apache Paimon Java 2.0's Mosaic behavior. The work is done when Mosaic is registered, projection and column-bucket metadata are supported, and C++ round-trip plus Java-produced compatibility tests pass.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Motivation
Apache Paimon Java 2.0 supports Mosaic, a hybrid format designed for wide tables using column bucketing and parallel I/O. Paimon C++ cannot currently access tables using this format.
Solution
Add Mosaic support to the Paimon C++ file-format SPI:
- Register the mosaic file-format factory.
- Implement compatible schema, reader, writer, and statistics handling.
- Support column projection and Mosaic column-bucket metadata.
- Add C++ round-trip tests and Java-produced file compatibility tests.
Anything else?
We will also support formats such as Lance and Vortex in the future.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- C++
- Stars
- 65
- Forks
- 29
- Avg merge
- 2d 30m
- Merged PRs (30d)
- 77
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/paimon-cpp
-
enhancement
apache/paimon-cpp#381 · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
apache/paimon-cpp#375 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
apache/paimon-cpp#369 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
apache/paimon-cpp#361 · 1 assignee ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 68/100
apache/paimon-cpp#347 · 1 assignee ·
All issues in apache/paimon-cpp
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·