[C++] ListArray::FromListView gives wrong nulls for sliced list-view arrays
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
Research direction
Start in cpp/src/arrow/array/array_nested.cc at ListFromListViewImpl and compare its validity-bitmap access with FlattenListViewArray, especially the offset handling. Reproduce the issue with the sliced list-view example, then add or update a regression test so ListArray::FromListView and LargeListArray::FromListView preserve the input nulls.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
ListArray::FromListView (and LargeListArray::FromListView) returns the wrong validity for a sliced list-view array with nulls.
auto views = arrow::json::ArrayFromJSONString(
arrow::list_view(arrow::int32()),
"[[1], [2], [3], [4], [5], [6], [7], [8], null, [10], [11]]")
.ValueOrDie();
auto sliced = std::static_pointer_cast<arrow::ListViewArray>(views->Slice(1));
auto lists =
arrow::ListArray::FromListView(*sliced, arrow::default_memory_pool()).ValueOrDie();
// is null (list view): 0 0 0 0 0 0 0 1 0 0
// is null (list): 0 0 1 1 1 1 1 1 1 1
Expected: the same nulls as the input.
In ListFromListViewImpl (cpp/src/arrow/array/array_nested.cc), the validity bitmap is read with list_view_data->GetValues<uint8_t>(0), which already advances the pointer by offset bytes. It is then passed to bit_util::GetBit(in_validity_bitmap, list_view_data->offset + i), which applies the offset again, in bits. FlattenListViewArray in the same file does this correctly with GetValues<uint8_t>(0, 0).
Found while trying to use FromListView in a compute kernel (GH-33295). Related: GH-51612, where the list_view to list cast could use FromListView. Reproduced on current main, Linux x86_64.
Component(s)
C++
- Dominant language
- C++
- Stars
- 17.2k
- Forks
- 4.3k
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 95
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow
-
Component: R Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/arrow#51695 · 1 comment ·
Maintainers usually reply within 2 days
-
[C++][Parquet] Plaintext-footer files written with AES_GCM_CTR_V1 record AES_GCM_V1 as the encryption algorithm and cannot be readPossibly taken @YusefSyed claimed this 1 day ago. OpenType: bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
[R] Expose ignore_extra_columns and pad_short_rows CSV parse optionsPossibly taken A pull request linked to this issue is open or already merged. OpenComponent: R good-first-issue Type: enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 2 days
-
Component: C++ Type: enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 2 days
-
Component: GLib
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 2 days
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
espressif/esp-matter#1874 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
opencv/opencv_contrib#4231 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
MiSTer-devel/Main_MiSTer#1341 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
linux-test-project/lcov#552 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 94/100
Maintainers usually reply within 1 day