Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

CFFI backend: crash when ZstdDecompressor instances share a ZstdCompressionDict

Abierto
  • ¿Algo no está bien?
#361 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 10 días

@pylaterreur ya está trabajando en esto.

Desde el 17/10/2026.

  • #362 de @pylaterreur — abierto

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python
Área
backend

Línea de trabajo

Start with ZstdCompressionDict._ddict in zstandard/backend_cffi.py and _ensure_dctx() in the same file, then compare how the C backend retains its DDict in c-ext/compressiondict.c. Run the reproducer with PYTHON_ZSTANDARD_IMPORT_POLICY=cffi and check for a regression test covering multiple decompressors sharing one dictionary. Done means repeated and concurrent use no longer crashes or raises a dictionary mismatch; the reporter says they plan to open a PR.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

With the CFFI backend, using one ZstdCompressionDict with more than one ZstdDecompressor can crash the interpreter, or fail with ZstdError: zstd decompress error: Dictionary mismatch. The C backend isn't affected.

This reproducer doesn't need threads:

import zstandard as zstd

samples = [b"sample %d " % i + b"data " * (i % 50) for i in range(1000)]
dict_data = zstd.train_dictionary(8192, samples)
data = b"".join(samples)
frame = zstd.ZstdCompressor(dict_data=dict_data).compress(data)

reader = zstd.ZstdDecompressor(dict_data=dict_data).stream_reader(frame)
zstd.ZstdDecompressor(dict_data=dict_data)  # another decompressor, same dictionary
assert reader.read() == data

With PYTHON_ZSTANDARD_IMPORT_POLICY=cffi, every run either segfaulted or raised ZstdError: zstd decompress error: Dictionary mismatch. I ran it with:

  • zstandard 0.25.0 (PyPI wheel) on CPython 3.14.8;
  • main (a2bb3c0) on CPython 3.15.0 and 3.15.0t.

With PYTHON_ZSTANDARD_IMPORT_POLICY=cext, it passes on all three. I also ran a stress test where 8 threads each create their own decompressors for one shared dictionary. With the CFFI backend it segfaults within a few hundred operations, both with and without the GIL.

Cause. ZstdCompressionDict._ddict in zstandard/backend_cffi.py (lines 2748–2766) is meant to create the DDict once and cache it in self.__dict__["_ddict"]. But property is a data descriptor, so it takes precedence over the instance __dict__ and the cached value is never read.

  • Every access creates a new DDict and stores it. That drops the previous DDict, and its ffi.gc destructor frees it.
  • ZstdDecompressor._ensure_dctx() runs when a decompressor is created and at the start of every operation. It passes self._dict_data._ddict to ZSTD_DCtx_refDDict() (line 4473), which only keeps a pointer.
  • So creating or using a second decompressor with the same dictionary frees the DDict that the first decompressor's context still points to.

The C backend creates the DDict once in ensure_ddict() (c-ext/compressiondict.c:148) and keeps it as long as the dictionary. The CFFI code has been like this since bef32a9 (2018), whose commit message says a DDict "can be used by multiple decompressors".

I have a fix that creates the DDict once, under a lock so that concurrent first uses don't create two, plus a regression test. I'll open a PR.

Prepared with the help of Claude Code.

Lenguaje dominante
C
Estrellas
642
Forks
118
Merge medio
9 d 3 h
PR fusionados (30 d)
6

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de indygreg/python-zstandard

Todos los issues de indygreg/python-zstandard

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.