BUG: DataStore resize aborts AddressSanitizer builds for allocations above 1 TiB

Abierto
#1,885 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 6 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
1-2 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cmake, cpp, github-actions

Línea de trabajo

Read src/simplnx/DataStructure/DataStore.hpp around resizeTuples() and its allocation failure handling, then inspect .github/workflows/asan.yml and the proposed test in test/DataArrayTest.cpp. Compare the two proposed fixes and check how the ASan test is configured. Done means the oversized resize reports error -6035, preserves the original store, and the regression test passes in the intended build configuration.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug

Problem

In an AddressSanitizer build, DataStore::resizeTuples() stops the whole process when the requested buffer is larger than the maximum allocation of AddressSanitizer (1 TiB on 64-bit platforms). The documented contract of resizeTuples() is that an allocation failure returns error -6035 and keeps the prior store. A filter or test that requests such a size therefore aborts the nightly asan CI job instead of reporting an error.

For example, a DataStore<uint8> with 4 values that is resized to 2^41 tuples (2 TiB) aborts with AddressSanitizer: allocation-size-too-big. This size does not overflow usize, so an overflow check does not catch it.

The nightly workflow (.github/workflows/asan.yml) does not set ASAN_OPTIONS, so AddressSanitizer uses its default allocator_may_return_null=0.

Occurs at develop commit 547f28a75e6325358fe7823e6dcb00defda81cd1.

Steps to Reproduce

  1. Configure a build of develop with AddressSanitizer: CMAKE_CXX_FLAGS="-O1 -fsanitize=address -fno-omit-frame-pointer", and -fsanitize=address in CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS. Do not set ASAN_OPTIONS.
  2. Add the test in Verification Unit Test to test/DataArrayTest.cpp, and build simplnx_test.
  3. Make a DataStore<uint8> with the tuple shape {4}, the component shape {1} and 4 values.
  4. Call resizeTuples({usize{1} << 41}).
  5. AddressSanitizer reports allocation-size-too-big and the process aborts.

Cause

resizeTuples() allocates the new buffer with array new and expects std::bad_alloc for a failed allocation:

https://github.com/BlueQuartzSoftware/simplnx/blob/547f28a75e6325358fe7823e6dcb00defda81cd1/src/simplnx/DataStructure/DataStore.hpp#L293

std::unique_ptr<value_type[]> data(new value_type[newSize]);

https://github.com/BlueQuartzSoftware/simplnx/blob/547f28a75e6325358fe7823e6dcb00defda81cd1/src/simplnx/DataStructure/DataStore.hpp#L311-L314

} catch(const std::exception& exception)
{
  return MakeErrorResult(-6035, fmt::format("DataStore resize to shape [{}] failed: {}", fmt::join(tupleShape, ", "), exception.what()));
}

With allocator_may_return_null=0, AddressSanitizer does not return from the allocation and does not throw. It reports a fatal error and aborts, so the catch block never runs. The DataStore constructors allocate in the same way (DataStore.hpp L101).

Verification Unit Test

File: test/DataArrayTest.cpp. The test runs only in an AddressSanitizer build. Without AddressSanitizer, the 2 TiB request can succeed, and the resize would then write 2 TiB.

Add this constant to the anonymous namespace at the top of the file, after k_BuildDir:

// GCC defines __SANITIZE_ADDRESS__ under -fsanitize=address; Clang reports it through __has_feature.
#if defined(__SANITIZE_ADDRESS__)
constexpr bool k_AddressSanitizerEnabled = true;
#elif defined(__has_feature)
#if __has_feature(address_sanitizer)
constexpr bool k_AddressSanitizerEnabled = true;
#else
constexpr bool k_AddressSanitizerEnabled = false;
#endif
#else
constexpr bool k_AddressSanitizerEnabled = false;
#endif
TEST_CASE("DataStore resize reports an allocation above the AddressSanitizer limit as an error", "[simplnx][DataStore]")
{
  if constexpr(k_AddressSanitizerEnabled)
  {
    DataStore<uint8> dataStore(ShapeType{4}, ShapeType{1}, 0);
    const std::array<uint8, 4> expectedValues = {10, 20, 30, 40};
    for(usize valueIndex = 0; valueIndex < expectedValues.size(); ++valueIndex)
    {
      dataStore[valueIndex] = expectedValues[valueIndex];
    }
    const uint8* const originalDataPtr = dataStore.data();
    // 2^41 one-byte values (2 TiB) fit usize and the array-new length limit, but exceed the
    // 1 TiB maximum allocation of AddressSanitizer on 64-bit platforms.
    const ShapeType oversizedTupleShape = {usize{1} << 41};

    const Result<> resizeResult = dataStore.resizeTuples(oversizedTupleShape);
    REQUIRE(resizeResult.invalid());
    REQUIRE_FALSE(resizeResult.errors().empty());
    REQUIRE(resizeResult.errors()[0].code == -6035);
    REQUIRE(dataStore.data() == originalDataPtr);
    REQUIRE(dataStore.getTupleShape() == ShapeType{4});
    REQUIRE(dataStore.getNumberOfTuples() == 4);
    REQUIRE(dataStore.getSize() == 4);
    for(usize valueIndex = 0; valueIndex < expectedValues.size(); ++valueIndex)
    {
      REQUIRE(dataStore[valueIndex] == expectedValues[valueIndex]);
    }
  }
  else
  {
    // Without AddressSanitizer the 2 TiB request can succeed, and the resize would then fill it.
    WARN("Skipped: this test runs only in an AddressSanitizer build.");
    SUCCEED("AddressSanitizer is not enabled.");
  }
}

Command (from the AddressSanitizer build directory):

ctest -R "^DataStore resize reports an allocation above the AddressSanitizer limit as an error$" --output-on-failure

Failure on develop (macOS arm64, AppleClang, AddressSanitizer):

==7362==ERROR: AddressSanitizer: requested allocation size 0x20000000000 (0x20000001000 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
    #0 0x0001099b81e0 in _Znam+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x501e0)
    #1 0x000104369854 in nx::core::DataStore<unsigned char>::resizeTuples(std::__1::vector<unsigned long, std::__1::allocator<unsigned long>> const&)+0x3f0 (simplnx_test:arm64+0x1002a1854)
==7362==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big (simplnx_test:arm64+0x1002a1854) in nx::core::DataStore<unsigned char>::resizeTuples(...)
==7362==ABORTING
DataArrayTest.cpp:863: FAILED:
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

In the Release build without AddressSanitizer, the same test is skipped and passes without an allocation.

Proposed Fix

Two approaches are possible. The choice is still open.

  • Configure the CI job: set ASAN_OPTIONS=allocator_may_return_null=1 for the test step of .github/workflows/asan.yml (or in a ci-asan test preset). AddressSanitizer then returns null for a too-large request, array new throws std::bad_alloc, and the existing catch returns -6035. This changes only the CI configuration.
  • Limit the size in the library: define a maximum supported allocation size for DataStore and check the byte count against it before the allocation in resizeTuples() and in the constructors. This gives the same result with and without AddressSanitizer, but it adds a new limit to the public behavior.

Notes

  • PR #1834 adds overflow checks to DataStore::resizeTuples(), and #1883 covers the constructors. Those checks catch a byte count that does not fit in usize, but not a valid size above the AddressSanitizer limit.
  • The 1 TiB limit is the AddressSanitizer maximum for 64-bit platforms in LLVM (kMaxAllowedMallocSize). The report above confirms it on macOS arm64. It was not checked on Linux or with GCC.
Lenguaje dominante
C++
Estrellas
18
Forks
15
Merge medio
4 d 23 h
PR fusionados (30 d)
42

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de BlueQuartzSoftware/simplnx

Todos los issues de BlueQuartzSoftware/simplnx

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.