Keyed JOIN/ANTIJOIN/SEMIJOIN dereference NULL columns for a nonempty zero-column input
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start with col_op_resolve_key and the join, antijoin, semijoin, and differential join paths in wirelog/columnar/join.c; compare the existing left-side guards and the range checks in join_pipeline.c. Read test_nonempty_nullary_left_cleanup and test_diff_nonempty_nullary_left_cleanup in tests/test_join_batch_resume.c, then run the relevant tests under ASan/UBSan. Done means the chosen behavior is covered for owned and borrowed inputs across the listed batch settings, empty-input behavior is preserved, and the session remains reusable.
Written by the indexing model from the issue text.
Description
A keyed JOIN, differential JOIN, ANTIJOIN or SEMIJOIN over a nonempty zero-column (nullary) input on the right, and an ANTIJOIN or SEMIJOIN over a nonempty zero-column left, dereferences the input's NULL column array and crashes. Follow-up of #2202, which fixed the differential JOIN's left side.
Why
col_op_resolve_key (wirelog/columnar/join.c) falls back to column 0 when a key name is not found. For a relation with ncols == 0, columns is NULL, so the first probe or hash of a row reads columns[0][row]. (column_types is NULL as well, and col_join_key_types_compatible then takes its INT64 default, so the type check itself does not read out of range.)
Measured (operator level, ASan/UBSan: at e3580a61, re-measured on the #2202 branch)
| Operator | nonempty 0-col left (key_count 1) | nonempty 0-col right (key_count 1) |
|---|---|---|
| JOIN | EINVAL (3be9ba28) |
NULL load in col_join_hash_rel_keys, knob off and at WIRELOG_JOIN_BATCH_BYTES=224, even with an empty left |
| JOIN_DIFF | EINVAL since #2202 |
SEGV in the one-shot hash (col_join_hash_rel_keys) with the knob off, and with an empty left also at 224 and 224 strict (an empty input is joined one-shot since #2202); with a nonempty left at 224 or 224 strict: EINVAL from the producer's key-range check since #2202 |
| ANTIJOIN | SEGV (join.c ~2425 via antijoin_emit) |
SEGV |
| SEMIJOIN | SEGV (join.c ~2425 via semijoin_emit) |
SEGV (planning measurement at e3580a61; not re-probed) |
An empty zero-column input returns rc 0 with the operator's empty-input result (measured knob off and at 224 without strict, unfiltered and not materialized): JOIN, JOIN_DIFF and SEMIJOIN give no rows, and ANTIJOIN over an empty right keeps its left rows. The pipeline sites in join_pipeline.c range-check key indices before the type check and are not affected. No plan-level route was found that produces a keyed join over a nonempty zero-column input; this is operator-level hardening.
To decide
- What an existence join (ANTIJOIN/SEMIJOIN) over a nullary keyed input should return:
EINVALlike JOIN, or a defined result. - Whether the right-side check belongs before the right-operand selection (missing / forced-empty right still yield empty) and before the right filter, mirroring the left-side guard.
- The differential JOIN's one-shot path builds a differential arrangement keyed on column 0 of a zero-column right (measured: one arrangement, 0 rows indexed).
test_diff_operator_empty_input_nothing_to_bound(#2202) asserts that this arrangement entry exists for an empty zero-column right; that assertion may need to change if the fix stops building it.
Tests
Model on test_nonempty_nullary_left_cleanup and test_diff_nonempty_nullary_left_cleanup (tests/test_join_batch_resume.c). For each operator: nonempty zero-column right (and left for ANTIJOIN/SEMIJOIN), knob off, 224 and 224 strict, owned and borrowed inputs: the chosen rc, stack and reservations unchanged, no fallback recorded, session reusable; empty zero-column inputs keep their current empty-input results. Run under ASan/UBSan.
- Dominant language
- C
- Stars
- 18
- Forks
- 2
- Avg merge
- 6h 57m
- Merged PRs (30d)
- 249
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from semantic-reasoning/wirelog
-
Doc: insert executor and incremental consolidation leave sorted runs, not globally sorted headsOpen
Difficulty 1/5 Under an hour Newbie friendliness 83/100
semantic-reasoning/wirelog#2133 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
semantic-reasoning/wirelog#2042 ·
Maintainers usually reply within 1 day
-
p2-medium
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
semantic-reasoning/wirelog#1914 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
semantic-reasoning/wirelog#2201 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 8/100
semantic-reasoning/wirelog#2186 ·
Maintainers usually reply within 1 day
All issues in semantic-reasoning/wirelog
Similar issues
-
Go device SDK: Whisper transcriber keeps buffering and transcribing audio appended after Stop()Possibly taken @apoorvdarshan claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
BasedHardware/omi#22770 ·
Maintainers usually reply within 1 day
-
severity: low
Difficulty 1/5 1-3 hours Newbie friendliness 89/100
luainkernel/lunatik#1980 ·
Maintainers usually reply within 1 day
-
Investigation-Finding Issue-Bug Needs-Author-Feedback Needs-Triage Product-Command Palette
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
microsoft/PowerToys#51421 · 1 comment ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
EchoTools/nevr-runtime#698 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
FujiNetWIFI/fujinet-firmware#2079 ·
Maintainers usually reply within 1 day