Detect Int64 SUM overflow on the CUDA aggregate path instead of wrapping
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 57/100
Research direction
Read crates/oxidelake-device/src/cuda/ops.rs and kernels/cuda/aggregation.cu to trace the CUDA SUM path, then compare it with the CPU overflow handling in crates/oxidelake-device/src/cpu/kernels.rs. Add the specified overflow fixture to conformance and run it against CPU, CUDA, and stock DataFusion; done means all three error and the “by design” comment is removed.
Written by the indexing model from the issue text.
Description
Today — crates/oxidelake-device/src/cuda/ops.rs:10-11: "Divergence from the CPU reference, by design: device SUM over Int64 wraps on overflow where the CPU path reports an error." The kernel uses atomicAdd((unsigned long long*)&sums[slot], …) (kernels/cuda/aggregation.cu:125); the CPU path errors with "SUM overflowed Int64" (crates/oxidelake-device/src/cpu/kernels.rs:439-441). Conformance has no overflow case (grep -n overflow conformance.rs → 0).
Why it is worth fixing — a documented wrong answer is still a wrong answer, and DataFusion's stock aggregate errors here. Under the equality-with-stock-DataFusion standard the conformance suite enforces, this is the one divergence left in by design.
Fix — accumulate an overflow flag on device (check __builtin_add_overflow-style before the atomic, or keep a per-slot high word and detect carry) and return EngineError::Execution("SUM overflowed Int64") on the host; add an overflow row (i64::MAX - 1 twice in one group) to conformance.
Done when — CPU, CUDA (GPU job) and stock DataFusion all error on the fixture; the "by design" comment is removed.
- Dominant language
- Rust
- Stars
- 1
- Forks
- 2
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 7
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from vyncint/oxidelake
-
dependencies documentation
Difficulty 2/5 1-3 hours Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 38/100
Maintainers usually reply within 1 day
-
Prepare the Ballista 55 / DataFusion 55 / arrow 59 chain bump ahead of the release that fixes thriftOpenenhancement
Difficulty 5/5 Over a week Newbie friendliness 32/100
vyncint/oxidelake#40 · 2 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
All issues in vyncint/oxidelake
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
storytold/craft-libs#471 ·
Maintainers usually reply within 1 day
-
enhancement parity priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
storytold/gridcraft#3053 · 1 comment ·
Maintainers usually reply within 2 days
-
bug file-compat parity priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 2 days
-
i18n multiple issues parity priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
storytold/vectorcraft#4943 · 3 comments ·
Maintainers usually reply within 1 day