[Bug] Nullability validation incorrectly checks children masked by null parents
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 58/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- cpp
- Domain
- data-engineering
Research direction
Start by locating ArrowUtils::CheckNullabilityMatch and the tests covering nested STRUCT, LIST, MAP, and VECTOR arrays. Verify cases where null parents contain physically null children, and cases where valid parents expose null children; done means masked children are ignored while visible non-nullable children are rejected.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Paimon-cpp version
master
Minimal reproduce step
ArrowUtils::CheckNullabilityMatch recursively checks complete Arrow child arrays.
For nested Arrow types, a null parent may still have physical child values. These
children are logically hidden and should not participate in nullability validation.
However, if a hidden child is null and its field is non-nullable, the current
implementation rejects the batch.
This affects nested STRUCT, LIST, MAP, and VECTOR fields and is inconsistent with
Java's logical row semantics.
What doesn't meet your expectations?
Expected behavior:
- Ignore child values masked by a null parent.
- Reject null children when the parent is valid and the child is logically visible.
Anything else?
No response
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- C++
- Stars
- 65
- Forks
- 29
- Avg merge
- 2d 30m
- Merged PRs (30d)
- 77
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/paimon-cpp
-
enhancement
apache/paimon-cpp#381 · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
apache/paimon-cpp#375 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
apache/paimon-cpp#369 · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 45/100
apache/paimon-cpp#361 · 1 assignee ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 68/100
apache/paimon-cpp#347 · 1 assignee ·
All issues in apache/paimon-cpp
Similar issues
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 88/100
QuantStack/git2cpp#187 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
mlcommons/mobile_app_open#1182 ·
-
Needs-Triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
microsoft/winget-cli#6547 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·