Stack buffer overflow in FASTCOVER_hashPtrToIndex when training with many 1-byte samples

Đọc lib/dictBuilder/fastcover.c tại FASTCOVER_hashPtrToIndex và FASTCOVER_selectSegment, sau đó tái hiện vấn đề thông qua ZDICT_trainFromBuffer bằng testcase.cpp được cung cấp dưới AddressSanitizer. Hoàn thành khi tám mẫu một byte không còn gây ra stack-buffer-overflow và lệnh gọi huấn luyện hoàn tất với kết quả hợp lệ hoặc mã lỗi.

Hi, there is a buffer overflow (read) reachable from ZDICT_trainFromBuffer caused by using many small samples.

Tested on the most recent commit a8f732e5.

(found via automated fuzzing)

Specifically, the issue here seems to be an edge case with:

const size_t sizes[8] = {1,1,1,1,1,1,1,1};

Variants like the following do not crash, and instead return an error code:

const size_t sizes[4] = {2,2,2,2};
const size_t sizes[2] = {4,4};
const size_t sizes[1] = {8};

testcase.cpp

#include <cstdio>
#include <cstddef>
#include <vector>
extern "C" {
#include "zstd.h"
}
extern "C" {
#include "zdict.h"
}
int main(){
  // 8 total bytes, split into 8 one-byte samples
  const char data[8] = {'A','B','C','D','E','F','G','H'};
  const size_t sizes[8] = {1,1,1,1,1,1,1,1};
  std::vector<unsigned char> dict(1024);
  // Triggers stack-buffer-overflow in FASTCOVER_hashPtrToIndex
  size_t r = ZDICT_trainFromBuffer(dict.data(), dict.size(), data, sizes, 8);
  (void)r;
  return 0;
}

crash report

{
  "Date": "2025-09-25T03:47:40.709765+00:00",
  "Uname": "Linux 9d32adc15194 5.15.0-140-generic #150-Ubuntu SMP Sat Apr 12 06:00:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux",
  "OS": "Ubuntu",
  "OSRelease": "22.04",
  "Architecture": "amd64",
  "ExecutablePath": "/tmp/tmp_ihq7gc8/reproducer",
  "ProcEnviron": [
    "PKG_CONFIG_PATH=/fuzz/install/lib/pkgconfig",
    "LIBAFL_EDGES_MAP_SIZE=800000",
    "PWD=/fuzz/workspace",
    "CXX=gf_libafl_cxx",
    "GRAPHFUZZ_USE_ASAN=1",
    "HOME=/root",
    "ASAN_OPTIONS=hard_rss_limit_mb=1024:detect_leaks=0",
    "TERM=xterm-256color",
    "SHLVL=1",
    "LD_LIBRARY_PATH=/fuzz/install/lib",
    "PATH=/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "CC=gf_libafl_cc",
    "DEBIAN_FRONTEND=noninteractive",
    "OLDPWD=/fuzz/src/lib",
    "_=/usr/local/bin/agfi"
  ],
  "ProcCmdline": "/tmp/tmp_ihq7gc8/reproducer",
  "Stdin": "",
  "ProcStatus": [],
  "ProcMaps": [],
  "ProcFiles": [],
  "NetworkConnections": [],
  "CrashSeverity": {
    "Type": "NOT_EXPLOITABLE",
    "ShortDescription": "stack-buffer-overflow(read)",
    "Description": "Stack buffer overflow",
    "Explanation": "The target reads data past the end, or before the beginning, of the intended stack buffer."
  },
  "Stacktrace": [
    "    #0 0x5555558c28e6 in FASTCOVER_hashPtrToIndex /fuzz/src/lib/dictBuilder/fastcover.c",
    "    #1 0x5555558c28e6 in FASTCOVER_selectSegment /fuzz/src/lib/dictBuilder/fastcover.c:174:24",
    "    #2 0x5555558c28e6 in FASTCOVER_buildDictionary /fuzz/src/lib/dictBuilder/fastcover.c:421:31",
    "    #3 0x5555558c479b in FASTCOVER_tryParameters /fuzz/src/lib/dictBuilder/fastcover.c:493:25",
    "    #4 0x5555558c39d2 in ZDICT_optimizeTrainFromBuffer_fastCover /fuzz/src/lib/dictBuilder/fastcover.c:737:11",
    "    #5 0x55555566c326 in ZDICT_trainFromBuffer /fuzz/src/lib/dictBuilder/zdict.c:1124:12",
    "    #6 0x55555565e57a in main /tmp/tmp_ihq7gc8/reproducer.cpp:16:14",
    "    #7 0x7ffff7a6dd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16",
    "    #8 0x7ffff7a6de3f in __libc_start_main csu/../csu/libc-start.c:392:3",
    "    #9 0x555555583344 in _start (/tmp/tmp_ihq7gc8/reproducer+0x2f344) (BuildId: 6d5af358a3f4a90aea453373a678701b4d81d589)"
  ],
  "Registers": {},
  "Disassembly": [],
  "Package": "",
  "PackageVersion": "",
  "PackageArchitecture": "",
  "PackageDescription": "",
  "AsanReport": [
    "==156==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffff5e00028 at pc 0x5555558c28e7 bp 0x7fffffffe5d0 sp 0x7fffffffe5c8",
    "READ of size 8 at 0x7ffff5e00028 thread T0",
    "    #0 0x5555558c28e6 in FASTCOVER_hashPtrToIndex /fuzz/src/lib/dictBuilder/fastcover.c",
    "    #1 0x5555558c28e6 in FASTCOVER_selectSegment /fuzz/src/lib/dictBuilder/fastcover.c:174:24",
    "    #2 0x5555558c28e6 in FASTCOVER_buildDictionary /fuzz/src/lib/dictBuilder/fastcover.c:421:31",
    "    #3 0x5555558c479b in FASTCOVER_tryParameters /fuzz/src/lib/dictBuilder/fastcover.c:493:25",
    "    #4 0x5555558c39d2 in ZDICT_optimizeTrainFromBuffer_fastCover /fuzz/src/lib/dictBuilder/fastcover.c:737:11",
    "    #5 0x55555566c326 in ZDICT_trainFromBuffer /fuzz/src/lib/dictBuilder/zdict.c:1124:12",
    "    #6 0x55555565e57a in main /tmp/tmp_ihq7gc8/reproducer.cpp:16:14",
    "    #7 0x7ffff7a6dd8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16",
    "    #8 0x7ffff7a6de3f in __libc_start_main csu/../csu/libc-start.c:392:3",
    "    #9 0x555555583344 in _start (/tmp/tmp_ihq7gc8/reproducer+0x2f344) (BuildId: 6d5af358a3f4a90aea453373a678701b4d81d589)",
    "",
    "Address 0x7ffff5e00028 is located in stack of thread T0 at offset 40 in frame",
    "    #0 0x55555565e44f in main /tmp/tmp_ihq7gc8/reproducer.cpp:10",
    "",
    "  This frame has 2 object(s):",
    "    [32, 40) 'data' (line 12) <== Memory access at offset 40 overflows this variable",
    "    [64, 128) 'sizes' (line 13)",
    "HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork",
    "      (longjmp and C++ exceptions *are* supported)",
    "SUMMARY: AddressSanitizer: stack-buffer-overflow /fuzz/src/lib/dictBuilder/fastcover.c in FASTCOVER_hashPtrToIndex",
    "Shadow bytes around the buggy address:",
    "  0x7ffff5dffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5dffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5dffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5dfff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5dfff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "=>0x7ffff5e00000: f1 f1 f1 f1 00[f2]f2 f2 00 00 00 00 00 00 00 00",
    "  0x7ffff5e00080: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5e00100: f1 f1 f1 f1 00 00 00 00 00 00 f2 f2 f2 f2 00 00",
    "  0x7ffff5e00180: 00 f2 f2 f2 f2 f2 f8 f8 f8 f3 f3 f3 f3 f3 f3 f3",
    "  0x7ffff5e00200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "  0x7ffff5e00280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00",
    "Shadow byte legend (one shadow byte represents 8 application bytes):",
    "  Addressable:           00",
    "  Partially addressable: 01 02 03 04 05 06 07",
    "  Heap left redzone:       fa",
    "  Freed heap region:       fd",
    "  Stack left redzone:      f1",
    "  Stack mid redzone:       f2",
    "  Stack right redzone:     f3",
    "  Stack after return:      f5",
    "  Stack use after scope:   f8",
    "  Global redzone:          f9",
    "  Global init order:       f6",
    "  Poisoned by user:        f7",
    "  Container overflow:      fc",
    "  Array cookie:            ac",
    "  Intra object redzone:    bb",
    "  ASan internal:           fe",
    "  Left alloca redzone:     ca",
    "  Right alloca redzone:    cb",
    "==156==ABORTING"
  ],
  "MsanReport": [],
  "UbsanReport": [],
  "LuaReport": [],
  "PythonReport": [],
  "GoReport": [],
  "JavaReport": [],
  "RustReport": [],
  "JsReport": [],
  "CSharpReport": [],
  "CrashLine": "/fuzz/src/lib/dictBuilder/fastcover.c",
  "Source": []
}