[Feature] Support Vortex file format
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Domain
- data-engineering, databases
Research direction
Start by reading the Java reference module at paimon-vortex and the Mosaic Rust FFI patterns in issue #238. Map the C++ file-format SPI requirements to the requested schema, reader, writer, projection, predicate, and row-selection behavior. Done means C++ round-trip tests and Java-produced file compatibility tests cover supported and unsupported logical types.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Motivation
Apache Paimon Java supports Vortex through the paimon-vortex module, a Rust-based columnar file format with extensible compression encodings and lazily loaded summary statistics. Paimon C++ cannot currently access tables using this format.
Solution
Add Vortex support to the Paimon C++ file-format SPI:
- Register the vortex file-format factory.
- Implement compatible schema, reader, and writer handling.
- Support column projection, predicate pushdown, and precise row selection.
- Validate the same supported and unsupported logical types as Java.
- Add C++ round-trip tests and Java-produced file compatibility tests.
Anything else?
Vortex is implemented in Rust, so the integration can follow the Mosaic Rust FFI patterns (#238).
Java reference module: https://github.com/apache/paimon/tree/master/paimon-vortex
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- C++
- Stars
- 65
- Forks
- 29
- Avg merge
- 2d 30m
- Merged PRs (30d)
- 77
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/paimon-cpp
-
enhancement
apache/paimon-cpp#381 · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
apache/paimon-cpp#375 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
apache/paimon-cpp#369 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
apache/paimon-cpp#361 · 1 assignee ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 68/100
apache/paimon-cpp#347 · 1 assignee ·
All issues in apache/paimon-cpp
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·