ZSTD_compressSequencesAndLiterals() does not enforce its documented litCapacity >= litSize + 8 contract — the implemented check is 8 bytes weaker than the API documentation, its own error message, and the introducing commit
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 88/100
Research direction
Start at the litCapacity check in lib/compress/zstd_compress.c:8128 and compare it with the contract in lib/zstd.h. Build with make -C lib, run the reproducer described in the issue, and verify that capacities below litSize + 8 are rejected while litSize + 8 still succeeds.
Written by the indexing model from the issue text.
Description
Describe the bug
ZSTD_compressSequencesAndLiterals() does not enforce its own documented
litBufCapacity >= litSize + 8 contract: the implemented check in
lib/compress/zstd_compress.c:8128 only tests litCapacity < litSize,
i.e. it is 8 bytes weaker than the contract stated in three independent
places:
- API documentation —
lib/zstd.hline 1700:@litBufCapacity must be at least 8 bytes larger than @litSize. - The introducing commit —
b7a9e69d("added parameter litCapacity
to ZSTD_compressSequencesAndLiterals() to enforce the litCapacity >=
litSize+8 condition.") - The check's own error message —
"literals buffer is not large enough: must be at least 8 bytes larger
than litSize (risk of read out-of-bound)"
/* lib/compress/zstd_compress.c:8126-8130 */
if (litCapacity < litSize) { /* <-- should be: litCapacity < litSize + 8 */
RETURN_ERROR(workSpace_tooSmall,
"literals buffer is not large enough: "
"must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
}
Calls that violate the documented contract by up to 8 bytes
(litSize <= litCapacity < litSize + 8) are silently accepted and produce
valid frames, instead of being rejected with the documented
ZSTD_error_workSpace_tooSmall.
To Reproduce
Steps to reproduce the behavior:
- Clone zstd and build the static library:
git clone https://github.com/facebook/zstd && cd zstd
make -C lib libzstd.a # or: make -C lib
- Save the following as
reproducer.cin the repo root:
/* reproducer.c — demonstrates that ZSTD_compressSequencesAndLiterals()
* accepts litCapacity values that violate its documented
* "at least 8 bytes larger than litSize" contract. */
#define ZSTD_STATIC_LINKING_ONLY
#include "zstd.h"
#include <stdio.h>
int main(void)
{
/* one explicit-delimiter block: 284 literals + a 16-byte match at
* offset 8, then 16 trailing literals carried by the delimiter */
ZSTD_Sequence seqs[2] = {
{ /*offset=*/8, /*litLength=*/284, /*matchLength=*/16, /*rep=*/0 },
{ /*offset=*/0, /*litLength=*/ 16, /*matchLength=*/ 0, /*rep=*/0 },
};
unsigned char literals[300 + 8]; /* +8 so the probe itself is memory-safe */
unsigned char dst[1024];
const size_t litSize = 300, srcSize = 316;
size_t i, r;
ZSTD_CCtx* cctx = ZSTD_createCCtx();
for (i = 0; i < litSize; i++) literals[i] = (unsigned char)('a' + (i % 26));
ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, 3);
ZSTD_CCtx_setParameter(cctx, ZSTD_c_blockDelimiters, ZSTD_sf_explicitBlockDelimiters);
/* documented minimum is litSize + 8; pass litSize + 7 on purpose */
r = ZSTD_compressSequencesAndLiterals(cctx, dst, sizeof(dst),
seqs, 2, literals, litSize,
litSize + 7, srcSize);
if (ZSTD_isError(r))
printf("litCapacity = litSize+7 : rejected (%s) -> contract enforced\n",
ZSTD_getErrorName(r));
else
printf("litCapacity = litSize+7 : ACCEPTED (%zu-byte frame) -> "
"documented contract NOT enforced\n", r);
ZSTD_freeCCtx(cctx);
return 0;
}
- Compile and run:
gcc -O2 -Ilib reproducer.c lib/libzstd.a -o reproducer -lpthread
./reproducer
- Scroll up on the log to the reproducer's output line:
litCapacity = litSize+7 : ACCEPTED (223-byte frame) -> documented contract NOT enforced
- See error: there is no error — the contract-violating call is
accepted and produces a valid, round-tripping 223-byte frame. The same
happens withlitCapacity = litSize(violating by the full 8 bytes).
With the one-line fix below, both calls are rejected with
ZSTD_error_workSpace_tooSmallwhile a compliant
litCapacity = litSize + 8call keeps working.
Expected behavior
ZSTD_compressSequencesAndLiterals() should reject litCapacity values
below litSize + 8 with ZSTD_error_workSpace_tooSmall, exactly as its
documentation (lib/zstd.h:1700), its introducing commit (b7a9e69d) and
its own error message all promise:
--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -8125,7 +8125,7 @@ ZSTD_compressSequencesAndLiterals(ZSTD_CCtx* cctx,
/* Transparent initialization stage, same as compressStream2() */
DEBUGLOG(4, "ZSTD_compressSequencesAndLiterals (dstCapacity=%zu)", dstCapacity);
assert(cctx != NULL);
- if (litCapacity < litSize) {
+ if (litCapacity < litSize + 8) {
RETURN_ERROR(workSpace_tooSmall, "literals buffer is not large enough: must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
}
FORWARD_IF_ERROR(ZSTD_CCtx_init_compressStream2(cctx, ZSTD_e_end, decompressedSize), "CCtx initialization failed");
Screenshots and charts
Not applicable — the reproducer output above shows the behavior
(ACCEPTED ... contract NOT enforced on current dev; after the one-line
fix the same call prints rejected (workSpace buffer is not large enough) -> contract enforced).
Desktop (please complete the following information):
- OS: Linux (x86-64)
- Version: Ubuntu 20.04
- Compiler: gcc 9.4.0
- Flags:
-O2for the reproducer; the fuller PoC (see Additional context) was additionally run with-g -O1 -fsanitize=address,undefined -fno-sanitize-recover=all - Other relevant hardware specs: n/a (no hardware dependency)
- Build system:
make -C lib(static librarylib/libzstd.a); library sources compiled directly in the PoC verification as well
Additional context
- Affected version: current
dev(verified byte-identical at
lib/compress/zstd_compress.c:8128ondev@d9c0c7e2, 2026-09-09).
I could not find an existing issue or PR covering this mismatch (closest:
open PR #4741 fixes a different validation gap in the same function). - Full verification PoC (attached): a three-phase test that (1) confirms a
compliantlitCapacity = litSize + 8call succeeds, (2) demonstrates both
litSize + 7andlitSizeviolations are accepted on the unpatched
build and rejected withZSTD_error_workSpace_tooSmallafter the
one-line fix, and (3) runs an ASan sweep of 360 configurations
(30 literal sizes x 3 data patterns x 4 levels, literals buffer of
exactlylitSizebytes) probing for reads inside the documented-but-
unenforced margin[litSize, litSize+8). - Honest impact statement: the sweep produced zero ASan reports —
literal reads are currently bounded bylitSizein the existing encoder
paths, so this is a contract-enforcement / defense-in-depth gap rather
than a demonstrated out-of-bounds read. The +8 slack exists because the
literal encoders historically used 8-byte-wide loads near the tail; the
check is the only guard between the caller and
ZSTD_entropyCompressSeqStore_internal(), and today it enforces none of
the slack that the documentation, the error message and commit
b7a9e69dall promise. - Suggested title for the issue: "ZSTD_compressSequencesAndLiterals() does
not enforce the documented litBufCapacity >= litSize + 8 contract (check
is missing the + 8)".
- Dominant language
- C
- Stars
- 27.9k
- Forks
- 2.6k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from facebook/zstd
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 1/5 1-3 hours Newbie friendliness 72/100
Similar issues
-
level/task module/gcp type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
docs
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
P3 sonic-vpp
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sonic-net/sonic-buildimage#29662 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
spack/spack-packages#6586 ·
-
category:port-update
Difficulty 2/5 1-3 hours Newbie friendliness 72/100