BUG: DataStore resize aborts AddressSanitizer builds for allocations above 1 TiB
Los mantenedores suelen responder en 6 días
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 45/100
Línea de trabajo
Read src/simplnx/DataStructure/DataStore.hpp around resizeTuples() and its allocation failure handling, then inspect .github/workflows/asan.yml and the proposed test in test/DataArrayTest.cpp. Compare the two proposed fixes and check how the ASan test is configured. Done means the oversized resize reports error -6035, preserves the original store, and the regression test passes in the intended build configuration.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
In an AddressSanitizer build, DataStore::resizeTuples() stops the whole process when the requested buffer is larger than the maximum allocation of AddressSanitizer (1 TiB on 64-bit platforms). The documented contract of resizeTuples() is that an allocation failure returns error -6035 and keeps the prior store. A filter or test that requests such a size therefore aborts the nightly asan CI job instead of reporting an error.
For example, a DataStore<uint8> with 4 values that is resized to 2^41 tuples (2 TiB) aborts with AddressSanitizer: allocation-size-too-big. This size does not overflow usize, so an overflow check does not catch it.
The nightly workflow (.github/workflows/asan.yml) does not set ASAN_OPTIONS, so AddressSanitizer uses its default allocator_may_return_null=0.
Occurs at develop commit 547f28a75e6325358fe7823e6dcb00defda81cd1.
Steps to Reproduce
- Configure a build of
developwith AddressSanitizer:CMAKE_CXX_FLAGS="-O1 -fsanitize=address -fno-omit-frame-pointer", and-fsanitize=addressinCMAKE_EXE_LINKER_FLAGSandCMAKE_SHARED_LINKER_FLAGS. Do not setASAN_OPTIONS. - Add the test in Verification Unit Test to
test/DataArrayTest.cpp, and buildsimplnx_test. - Make a
DataStore<uint8>with the tuple shape{4}, the component shape{1}and 4 values. - Call
resizeTuples({usize{1} << 41}). - AddressSanitizer reports
allocation-size-too-bigand the process aborts.
Cause
resizeTuples() allocates the new buffer with array new and expects std::bad_alloc for a failed allocation:
std::unique_ptr<value_type[]> data(new value_type[newSize]);
} catch(const std::exception& exception)
{
return MakeErrorResult(-6035, fmt::format("DataStore resize to shape [{}] failed: {}", fmt::join(tupleShape, ", "), exception.what()));
}
With allocator_may_return_null=0, AddressSanitizer does not return from the allocation and does not throw. It reports a fatal error and aborts, so the catch block never runs. The DataStore constructors allocate in the same way (DataStore.hpp L101).
Verification Unit Test
File: test/DataArrayTest.cpp. The test runs only in an AddressSanitizer build. Without AddressSanitizer, the 2 TiB request can succeed, and the resize would then write 2 TiB.
Add this constant to the anonymous namespace at the top of the file, after k_BuildDir:
// GCC defines __SANITIZE_ADDRESS__ under -fsanitize=address; Clang reports it through __has_feature.
#if defined(__SANITIZE_ADDRESS__)
constexpr bool k_AddressSanitizerEnabled = true;
#elif defined(__has_feature)
#if __has_feature(address_sanitizer)
constexpr bool k_AddressSanitizerEnabled = true;
#else
constexpr bool k_AddressSanitizerEnabled = false;
#endif
#else
constexpr bool k_AddressSanitizerEnabled = false;
#endif
TEST_CASE("DataStore resize reports an allocation above the AddressSanitizer limit as an error", "[simplnx][DataStore]")
{
if constexpr(k_AddressSanitizerEnabled)
{
DataStore<uint8> dataStore(ShapeType{4}, ShapeType{1}, 0);
const std::array<uint8, 4> expectedValues = {10, 20, 30, 40};
for(usize valueIndex = 0; valueIndex < expectedValues.size(); ++valueIndex)
{
dataStore[valueIndex] = expectedValues[valueIndex];
}
const uint8* const originalDataPtr = dataStore.data();
// 2^41 one-byte values (2 TiB) fit usize and the array-new length limit, but exceed the
// 1 TiB maximum allocation of AddressSanitizer on 64-bit platforms.
const ShapeType oversizedTupleShape = {usize{1} << 41};
const Result<> resizeResult = dataStore.resizeTuples(oversizedTupleShape);
REQUIRE(resizeResult.invalid());
REQUIRE_FALSE(resizeResult.errors().empty());
REQUIRE(resizeResult.errors()[0].code == -6035);
REQUIRE(dataStore.data() == originalDataPtr);
REQUIRE(dataStore.getTupleShape() == ShapeType{4});
REQUIRE(dataStore.getNumberOfTuples() == 4);
REQUIRE(dataStore.getSize() == 4);
for(usize valueIndex = 0; valueIndex < expectedValues.size(); ++valueIndex)
{
REQUIRE(dataStore[valueIndex] == expectedValues[valueIndex]);
}
}
else
{
// Without AddressSanitizer the 2 TiB request can succeed, and the resize would then fill it.
WARN("Skipped: this test runs only in an AddressSanitizer build.");
SUCCEED("AddressSanitizer is not enabled.");
}
}
Command (from the AddressSanitizer build directory):
ctest -R "^DataStore resize reports an allocation above the AddressSanitizer limit as an error$" --output-on-failure
Failure on develop (macOS arm64, AppleClang, AddressSanitizer):
==7362==ERROR: AddressSanitizer: requested allocation size 0x20000000000 (0x20000001000 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
#0 0x0001099b81e0 in _Znam+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x501e0)
#1 0x000104369854 in nx::core::DataStore<unsigned char>::resizeTuples(std::__1::vector<unsigned long, std::__1::allocator<unsigned long>> const&)+0x3f0 (simplnx_test:arm64+0x1002a1854)
==7362==HINT: if you don't care about these errors you may set allocator_may_return_null=1
SUMMARY: AddressSanitizer: allocation-size-too-big (simplnx_test:arm64+0x1002a1854) in nx::core::DataStore<unsigned char>::resizeTuples(...)
==7362==ABORTING
DataArrayTest.cpp:863: FAILED:
due to a fatal error condition:
SIGABRT - Abort (abnormal termination) signal
In the Release build without AddressSanitizer, the same test is skipped and passes without an allocation.
Proposed Fix
Two approaches are possible. The choice is still open.
- Configure the CI job: set
ASAN_OPTIONS=allocator_may_return_null=1for the test step of.github/workflows/asan.yml(or in aci-asantest preset). AddressSanitizer then returns null for a too-large request, arraynewthrowsstd::bad_alloc, and the existingcatchreturns -6035. This changes only the CI configuration. - Limit the size in the library: define a maximum supported allocation size for
DataStoreand check the byte count against it before the allocation inresizeTuples()and in the constructors. This gives the same result with and without AddressSanitizer, but it adds a new limit to the public behavior.
Notes
- PR #1834 adds overflow checks to
DataStore::resizeTuples(), and #1883 covers the constructors. Those checks catch a byte count that does not fit inusize, but not a valid size above the AddressSanitizer limit. - The 1 TiB limit is the AddressSanitizer maximum for 64-bit platforms in LLVM (
kMaxAllowedMallocSize). The report above confirms it on macOS arm64. It was not checked on Linux or with GCC.
- Lenguaje dominante
- C++
- Estrellas
- 18
- Forks
- 15
- Merge medio
- 4 d 23 h
- PR fusionados (30 d)
- 42
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de BlueQuartzSoftware/simplnx
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
BlueQuartzSoftware/simplnx#1987 ·
Los mantenedores suelen responder en 6 días
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
BlueQuartzSoftware/simplnx#1980 ·
Los mantenedores suelen responder en 6 días
-
Testing
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
BlueQuartzSoftware/simplnx#1979 ·
Los mantenedores suelen responder en 6 días
-
documentation
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
BlueQuartzSoftware/simplnx#1978 ·
Los mantenedores suelen responder en 6 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
BlueQuartzSoftware/simplnx#1945 ·
Los mantenedores suelen responder en 6 días
Todos los issues de BlueQuartzSoftware/simplnx
Issues similares
-
bug triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
26.04 bug dark-mode Easy Hack impress symptom: visual
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
CollaboraOnline/online#16421 · 1 comentario ·
-
area/cs bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
ydb-platform/ydb#59881 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 64/100
Los mantenedores suelen responder en 1 día
-
documentation enhancement lua server
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100