Eliminate redundant overflow checks and duplicate size calculations in `TrailingSliceLayout::size_for_elems` and `_with_elems` casts

Open
#3,901 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Refactor
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
performance

Research direction

Start with zerocopy/src/layout.rs, especially TrailingSliceLayout::size_for_elems and DstLayout::validate_cast_and_convert_metadata, then trace the cast path in zerocopy/src/pointer/inner.rs and the reference constructors in zerocopy/src/ref.rs. Use the existing layout and reference tests plus the named benchmarks in zerocopy/benches/ to compare behavior and generated assembly. Done means the overflow, padding, and duplicate metadata checks are removed without changing validity or size results.

Written by the indexing model from the issue text.

Description

Summary

While #3659 optimized TrailingSliceLayout::padding_for_elems (by masking self.elem_size & size_mask before multiplying by elems) and eliminated the elems * elem_size round-trip in DstLayout::validate_cast_and_convert_metadata when elem_size <= size_align, several closely related layout arithmetic and overflow-checking redundancies remain on main (70fa3872c8):

  1. TrailingSliceLayout::size_for_elems performs redundant overflow checks and uses usize::MAX instead of sharing max_slice_len's bound, and Ref::from_*_with_elems calls T::size_for_metadata(count) twice.
  2. DstLayout::validate_cast_and_convert_metadata emits a redundant round_down(unused_bytes, size_align) mask when elem_size > size_align and elem_size % size_align == 0 (and see also closed PR #3716 regarding elem_size > size_align).

1. Redundant overflow checks in TrailingSliceLayout::size_for_elems, PtrInner::try_cast_into, and Ref::from_*_with_elems

What happens on main

In zerocopy/src/layout.rs:427-491, TrailingSliceLayout::size_for_elems computes:

pub(crate) const fn size_for_elems(&self, elems: usize) -> Option<usize> {
    let max_trailing_bytes = match self.max_trailing_bytes(usize::MAX) {
        Some(max) => max,
        None => return None,
    };

    let trailing_size = match self.elem_size.checked_mul(elems) {
        Some(size) => size,
        None => return None,
    };

    if trailing_size > max_trailing_bytes {
        return None;
    }

    #[allow(clippy::arithmetic_side_effects)]
    let size = (self.offset + trailing_size) + self.padding_for_elems(elems);
    Some(size)
}

There are three compounding redundancies here:

  1. Single pre-multiplication bound suffices: When elem_size > 0, elems * elem_size <= max_trailing_bytes holds if and only if elems <= max_trailing_bytes / elem_size (which is a compile-time constant for a given type T!). Checking elems > max_trailing_bytes / elem_size before multiplying by elem_size makes both checked_mul and trailing_size > max_trailing_bytes redundant—replacing a runtime overflowing multiplication + two branches with a single comparison against a compile-time constant.
  2. Mismatch between isize::MAX (max_slice_len) and usize::MAX (size_for_elems):
    In PtrInner::try_cast_into (zerocopy/src/pointer/inner.rs:584-592), when explicit metadata Some(elems) is passed:
    • First, U::is_valid_metadata(elems) checks elems <= trailing.max_slice_len::<{ DstLayout::MAX_SIZE }>() (bounded by isize::MAX as usize).
    • Immediately afterward, U::size_for_metadata(elems) calls trailing.size_for_elems(elems), which checks self.elem_size.checked_mul(elems) and trailing_size <= self.max_trailing_bytes(usize::MAX) (bounded by usize::MAX).
  3. Duplicate T::size_for_metadata(count) call in Ref::from_*_with_elems:
    In zerocopy/src/ref.rs:738-749:
    let expected_len = match T::size_for_metadata(count) {
        Some(len) => len,
        None => return Err(SizeError::new(bytes).into()),
    };
    if bytes.len() < expected_len {
        return Err(SizeError::new(bytes).into());
    }
    let (bytes, remainder) = bytes.split_at(expected_len);
    Ptr::from_ref(bytes)
        .try_cast_into_no_leftover::<T, BecauseImmutable>(Some(count))
    
    Ref::from_bytes_with_elems, Ref::from_prefix_with_elems, and Ref::from_suffix_with_elems call T::size_for_metadata(count) up front, and then delegate to try_cast_into_no_leftover(Some(count)), which calls T::is_valid_metadata(count) and T::size_for_metadata(count) a second time.
Observed assembly in zerocopy/benches/

In zerocopy/benches/ref_type_from_bytes_with_elems_dynamic_size.x86-64 (offset = 4, elem_size = 2, align = 2), LLVM emits three separate bounds/overflow checks on count (rcx):

 add	rsi, -4
 lea	rdi, [rcx + rcx]
 test	rcx, rcx
 setns	r8b                                # Check 1: checked_mul(count, 2) overflow
 cmp	rdi, -5
 setb	dil                                # Check 2: trailing_size <= max_trailing_bytes(usize::MAX)
 and	dil, r8b
 movabs	r9, 4611686018427387902
 cmp	rcx, r9
 setb	r9b                                # Check 3: count <= max_slice_len(isize::MAX)

And in zerocopy/benches/ref_type_from_bytes_with_elems_dynamic_padding.x86-64 (offset = 9, elem_size = 3, align = 4), LLVM loads two different 64-bit magic constants (6148914691236517201 $= \lfloor(\text{usize::MAX} - 9)/3\rfloor$ and 3074457345618258599 $= \lfloor(\text{isize::MAX} - 9)/3\rfloor$) and checks cmp r8, -12 twice:

 movabs	r8, 6148914691236517201
 cmp	rcx, r8
 ja	.LBB0_4
 lea	r8, [rcx + 2*rcx]
 cmp	r8, -12
 ja	.LBB0_4
 ...
 movabs	r10, 3074457345618258599
 cmp	rcx, r10
 setbe	cl
 cmp	r8, -12
 setb	r8b

Similarly, in ref_from_bytes_with_elems_dynamic_padding.x86-64, ref_from_prefix_with_elems_dynamic_padding.x86-64, and ref_from_suffix_with_elems_dynamic_padding.x86-64, LLVM checks both count <= 3074457345618258599 (isize::MAX bound) and count <= 6148914691236517201 (usize::MAX bound) + cmp rcx, -12.

Suggested fix
  • In TrailingSliceLayout::size_for_elems(elems) (or a size_for_elems_bounded<const MAX_SIZE: usize>), check if elems > self.max_slice_len::<MAX_SIZE>() { return None; } first (where max_slice_len is a compile-time constant for a given TrailingSliceLayout), after which let trailing_size = self.elem_size * elems; and (self.offset + trailing_size) + self.padding_for_elems(elems) are guaranteed not to overflow without any further checked_mul or trailing_size > max_trailing_bytes checks.
  • In PtrInner::try_cast_into, avoid checking U::is_valid_metadata(elems) separately from U::size_for_metadata(elems) when both are bounded by DstLayout::MAX_SIZE (or have size_for_metadata enforce DstLayout::MAX_SIZE).
  • In Ref::from_bytes_with_elems, Ref::from_prefix_with_elems, and Ref::from_suffix_with_elems, delegate directly to Ptr::from_ref(bytes).try_cast_into(...) / try_cast_into_no_leftover(Some(count)) instead of pre-computing T::size_for_metadata(count) and splitting the slice beforehand.

2. DstLayout::validate_cast_and_convert_metadata when elem_size > size_align

In zerocopy/src/layout.rs:1766-1779:

let elems = max_slice_bytes / trailing.elem_size.get();
let unused_bytes = max_slice_bytes % trailing.elem_size.get();
let unused_aligned_bytes = round_down(unused_bytes, size_align);
let self_bytes = max_total_bytes - unused_aligned_bytes;
  • When elem_size % size_align == 0 (even if elem_size > size_align, e.g. #[repr(C)] struct S([u8; 4], [[u8; 6]]) with size_align = 2, elem_size = 6), both max_slice_bytes (= round_down(bytes_len - size_base, size_align)) and elem_size are multiples of size_align. Thus unused_bytes = max_slice_bytes % elem_size is already a multiple of size_align, so round_down(unused_bytes, size_align) emits an unnecessary & !(size_align - 1) mask when elem_size > size_align.
  • Additionally, as explored in closed PR #3716 ("Optimize validate_cast_and_convert_metadata"), for elem_size > size_align where elem_size is not a power of two, computing self_bytes from unused_bytes can cause LLVM to duplicate division/remainder work compared to reconstructing self_bytes from elems * elem_size.

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

Dominant language
Rust
Stars
2.6k
Forks
194
Avg merge
18h 15m
Merged PRs (30d)
14

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from google/zerocopy

All issues in google/zerocopy

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.