One-shot decoding refuses some skippable frames

Aperta Adatta ai principianti
  • Qualcosa non va?
#4,882 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
72/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
c
Ambito
data

Direzione di ricerca

Il controllo di overflow si trova in readSkippableFrameSize in lib/decompress/zstd_decompress.c, intorno alle righe 591-595; le altre funzioni citate nell'issue (ZSTD_findFrameCompressedSize, ZSTD_decompress, ZSTD_findDecompressedSize, ZSTD_decompressBound, ZSTD_readSkippableFrame) la chiamano tutte. Compila il PoC contro lib/libzstd.a per vedere il rifiuto attuale, poi confronta il risultato con il percorso di streaming. È completato quando il frame con Frame_Size 0xFFFFFFF8 viene accettato dalle funzioni one-shot e di dimensione del frame con i valori di ritorno elencati nell'issue, e un frame troncato restituisce ancora srcSize_wrong.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Describe the bug
A skippable frame whose Frame_Size is between 0xFFFFFFF8 and 0xFFFFFFFF is refused with frameParameter_unsupported by ZSTD_findFrameCompressedSize and ZSTD_decompress. ZSTD_decompressStream skips the same frame without error. RFC 8878, section 3.1.2, allows User_Data of up to 2^32-1 bytes.

Cause: readSkippableFrameSize checks that sizeU32 + 8 fits in 32 bits (L594-L595), although it then computes the frame size in size_t. ZSTD_findDecompressedSize, ZSTD_decompressBound and ZSTD_readSkippableFrame go through the same function and refuse the frame too (output below).

To Reproduce
The PoC builds one skippable frame with Frame_Size = 0xFFFFFFF8 (a 4 GiB buffer from calloc).

  1. make -C lib libzstd.a
  2. cc -O2 -Ilib poc.c lib/libzstd.a -o poc && ./poc
  3. Output:
frame of 4294967296 bytes (User_Data 4294967288 bytes)
ZSTD_findFrameCompressedSize: Unsupported frame parameter
ZSTD_decompress: Unsupported frame parameter
ZSTD_decompressStream: 0, read 4294967296 of 4294967296

With -DSIZE=0xFFFFFFF0u, just below the limit, every call accepts the frame.

The same frame passed to the other functions that read a skippable frame's size (ZSTD_STATIC_LINKING_ONLY), on dev and v1.5.7:

ZSTD_findDecompressedSize: ZSTD_CONTENTSIZE_ERROR
ZSTD_decompressBound: ZSTD_CONTENTSIZE_ERROR
ZSTD_readSkippableFrame: Src size is incorrect
poc.c 
#include <stdio.h>
#include <stdlib.h>
#include "zstd.h"
#ifndef SIZE
#define SIZE 0xFFFFFFF8u
#endif

int main(void)
{
    size_t len = 8 + (size_t) SIZE;
    unsigned char *f = calloc(len, 1), out[1];
    if (f == NULL) { printf("no memory for %zu bytes\n", len); return 1; }
    f[0] = 0x50; f[1] = 0x2A; f[2] = 0x4D; f[3] = 0x18;                       /* Magic_Number 0x184D2A50 */
    f[4] = (unsigned char) SIZE; f[5] = (unsigned char) (SIZE >> 8);          /* Frame_Size, little-endian */
    f[6] = (unsigned char) (SIZE >> 16); f[7] = (unsigned char) (SIZE >> 24);
    size_t fs = ZSTD_findFrameCompressedSize(f, len);
    size_t d = ZSTD_decompress(out, sizeof out, f, len);
    printf("frame of %zu bytes (User_Data %u bytes)\n", len, SIZE);
    printf("ZSTD_findFrameCompressedSize: %s\n", ZSTD_isError(fs) ? ZSTD_getErrorName(fs) : "accepted");
    printf("ZSTD_decompress: %s\n", ZSTD_isError(d) ? ZSTD_getErrorName(d) : "accepted");
    ZSTD_DStream *zds = ZSTD_createDStream();
    ZSTD_inBuffer in = { f, len, 0 };
    ZSTD_outBuffer o = { out, sizeof out, 0 };
    size_t r = ZSTD_decompressStream(zds, &o, &in);
    printf("ZSTD_decompressStream: %s, read %zu of %zu\n", ZSTD_isError(r) ? ZSTD_getErrorName(r) : "0", in.pos, len);
    ZSTD_freeDStream(zds);
    free(f);
    return 0;
}

Expected behavior
One-shot decoding and the frame-size functions accept the frame, as streaming does: ZSTD_findFrameCompressedSize returns 4294967296, ZSTD_decompress, ZSTD_findDecompressedSize and ZSTD_decompressBound return 0, and ZSTD_readSkippableFrame returns the 4294967288 bytes of User_Data.

Desktop:

  • OS: Ubuntu 24.04.4 LTS
  • Version: dev 49cf51799, v1.5.7 and v1.4.9 (the check is the same in all three)
  • Compiler: gcc 13.3.0
  • Flags: default lib/Makefile flags, PoC built with -O2
  • Other relevant hardware specs: x86_64
  • Build system: Makefile

Additional context
Suggested fix: compare in size_t. srcSize >= 8 is checked just before, so neither side wraps, also with a 32-bit size_t. A truncated frame then gives srcSize_wrong, as it already does below the limit.

--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -591,12 +591,8 @@ static size_t readSkippableFrameSize(void const* src, size_t srcSize)
     RETURN_ERROR_IF(srcSize < ZSTD_SKIPPABLEHEADERSIZE, srcSize_wrong, "");
 
     sizeU32 = MEM_readLE32((BYTE const*)src + ZSTD_FRAMEIDSIZE);
-    RETURN_ERROR_IF((U32)(sizeU32 + ZSTD_SKIPPABLEHEADERSIZE) < sizeU32,
-                    frameParameter_unsupported, "");
-    {   size_t const skippableSize = skippableHeaderSize + sizeU32;
-        RETURN_ERROR_IF(skippableSize > srcSize, srcSize_wrong, "");
-        return skippableSize;
-    }
+    RETURN_ERROR_IF(sizeU32 > srcSize - skippableHeaderSize, srcSize_wrong, "");
+    return skippableHeaderSize + sizeU32;
 }
 
 /*! ZSTD_readSkippableFrame() :

Found by genproof.com Formal Verification Agent. Reviewed and approved by a human.

Lingua principale
C
Stelle
28.1k
Fork
2.7k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di facebook/zstd

Tutte le issue di facebook/zstd

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.