Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

CFFI backend: crash when ZstdDecompressor instances share a ZstdCompressionDict

未关闭
  • 哪里不对？
#361 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 10 天内回复

@pylaterreur 已经在做这个了。

开始于 2026年10月17日。

  • #362 来自 @pylaterreur —— 未关闭

评估

难度
3/5
预计耗时
1-2 天
新手友好度
45/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python
领域
backend

调研方向

Start with ZstdCompressionDict._ddict in zstandard/backend_cffi.py and _ensure_dctx() in the same file, then compare how the C backend retains its DDict in c-ext/compressiondict.c. Run the reproducer with PYTHON_ZSTANDARD_IMPORT_POLICY=cffi and check for a regression test covering multiple decompressors sharing one dictionary. Done means repeated and concurrent use no longer crashes or raises a dictionary mismatch; the reporter says they plan to open a PR.

由索引模型根据 Issue 内容生成。

描述

With the CFFI backend, using one ZstdCompressionDict with more than one ZstdDecompressor can crash the interpreter, or fail with ZstdError: zstd decompress error: Dictionary mismatch. The C backend isn't affected.

This reproducer doesn't need threads:

import zstandard as zstd

samples = [b"sample %d " % i + b"data " * (i % 50) for i in range(1000)]
dict_data = zstd.train_dictionary(8192, samples)
data = b"".join(samples)
frame = zstd.ZstdCompressor(dict_data=dict_data).compress(data)

reader = zstd.ZstdDecompressor(dict_data=dict_data).stream_reader(frame)
zstd.ZstdDecompressor(dict_data=dict_data)  # another decompressor, same dictionary
assert reader.read() == data

With PYTHON_ZSTANDARD_IMPORT_POLICY=cffi, every run either segfaulted or raised ZstdError: zstd decompress error: Dictionary mismatch. I ran it with:

  • zstandard 0.25.0 (PyPI wheel) on CPython 3.14.8;
  • main (a2bb3c0) on CPython 3.15.0 and 3.15.0t.

With PYTHON_ZSTANDARD_IMPORT_POLICY=cext, it passes on all three. I also ran a stress test where 8 threads each create their own decompressors for one shared dictionary. With the CFFI backend it segfaults within a few hundred operations, both with and without the GIL.

Cause. ZstdCompressionDict._ddict in zstandard/backend_cffi.py (lines 2748–2766) is meant to create the DDict once and cache it in self.__dict__["_ddict"]. But property is a data descriptor, so it takes precedence over the instance __dict__ and the cached value is never read.

  • Every access creates a new DDict and stores it. That drops the previous DDict, and its ffi.gc destructor frees it.
  • ZstdDecompressor._ensure_dctx() runs when a decompressor is created and at the start of every operation. It passes self._dict_data._ddict to ZSTD_DCtx_refDDict() (line 4473), which only keeps a pointer.
  • So creating or using a second decompressor with the same dictionary frees the DDict that the first decompressor's context still points to.

The C backend creates the DDict once in ensure_ddict() (c-ext/compressiondict.c:148) and keeps it as long as the dictionary. The CFFI code has been like this since bef32a9 (2018), whose commit message says a DDict "can be used by multiple decompressors".

I have a fix that creates the DDict once, under a lock so that concurrent first uses don't create two, plus a regression test. I'll open a PR.

Prepared with the help of Claude Code.

主要语言
C
星标
642
派生
118
平均合并
9 天 3 小时
30 天内合并 PR
6

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

indygreg/python-zstandard 的其他 Issue

查看 indygreg/python-zstandard 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。