`substring` errors when the bytes under a null `Utf8` slot would be cut inside a character

Open Beginner friendly
  • Is something wrong?
#11,590 1 comment 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@cakeni is already working on this.

Since Oct 17, 2026.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
data

Research direction

Start in arrow-string/src/substring.rs, in byte_substring, where check_char_boundary runs on every slot. string_view_substring already skips null slots before the same check, so it is the reference for the change. Reproduce with the nullif example in the issue, where the first slot is null and cuts inside a multi-byte character. Done when that case returns null for the slot instead of an error, non-null invalid boundaries still error, and the substring tests pass.

Written by the indexing model from the issue text.

Description

Describe the bug

substring on a StringArray or LargeStringArray checks char boundaries in every slot, including null slots. If a null slot covers a multi-byte character and the cut falls inside it, the whole call returns an error, although the output for that slot is null.

To Reproduce
use arrow_array::{BooleanArray, StringArray};
use arrow_select::nullif::nullif;
use arrow_string::substring::substring;

fn main() {
    let array = nullif(&StringArray::from(vec!["\u{e9}", "ab"]), &BooleanArray::from(vec![true, false])).unwrap();
    println!("{:?}", substring(&array, 1, None));
    // Err(ComputeError("The offset 1 is at an invalid utf-8 boundary."))
}
Expected behavior

Ok with [null, "b"]. Only non-null values should be able to cause the documented char boundary error.

Additional context

The Columnar format lets a null slot cover a non-empty range with undefined contents. byte_substring computes the new start and end of every slot through check_char_boundary, with no null check (substring.rs). string_view_substring skips null slots before the same check (substring.rs), so a StringViewArray with a null view over the same bytes returns [null, "b"].

nullif keeps the bytes of the slots it sets to null, so its output has this shape. Arrays from other producers through FFI or IPC can too.

Found with property-based tests that build arrays in every physical layout the Arrow format allows.

Dominant language
Rust
Stars
3.7k
Forks
1.4k
Avg merge
2d 8h
Merged PRs (30d)
309

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/arrow-rs

All issues in apache/arrow-rs

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.