Segfault: 4+ unchecked `PyBytes_AsStringAndSize` on user `read()` return uses uninitialized memory
维护者通常 10 天内回复
还没有人认领这个 Issue。
评估
调研方向
从 c-ext/compressor.c、c-ext/decompressor.c、c-ext/compressoriterator.c 和 c-ext/decompressoriterator.c 中列出的 PyBytes_AsStringAndSize 调用开始，然后检查 read_compressor_input 及其 decompressor 对应部分中的其他调用。使用提供的 BadSource 示例进行复现，并验证非 bytes 返回值会引发 TypeError，且在每条受影响的 streaming 路径上都不会发生崩溃或 assertion 中止。
由索引模型根据 Issue 内容生成。
描述
Summary
Several streaming paths call PyBytes_AsStringAndSize(result, &readBuffer, &readSize) on the return value of a caller-supplied source.read() and do not check the return value. When read() returns a non-bytes object (e.g., str, None, bytearray), readBuffer and readSize are left with their prior/uninitialized contents; the next ZSTD_compressStream2 call reads from those addresses. Observed: SEGV on release builds, _Py_CheckFunctionResult abort on debug builds.
Impact
- Severity: SEGV on release builds; assertion abort on debug builds.
- Reachability: Any caller-supplied
source.read()whose return is notbytes. A trivial wrapper around a text file or a mistakenly-returnedbytearraytriggers it. - Version: 0.25.0 (commit
7a77a75). - Platform: Confirmed Linux x86_64 / CPython 3.14 debug; bug is platform-independent.
Reproducers
SEGV on release — via copy_stream:
import zstandard, io
class BadSource:
def read(self, size):
return 'not bytes' # str, not bytes
comp = zstandard.ZstdCompressor()
comp.copy_stream(BadSource(), io.BytesIO())
# Segmentation fault
Assertion abort on debug — via iterator:
import zstandard
class BadSource:
def read(self, size):
return 'not bytes'
comp = zstandard.ZstdCompressor()
it = comp.read_to_iter(BadSource())
next(it)
# Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
# TypeError: expected bytes, str found
Root cause
PyBytes_AsStringAndSize returns -1 and sets a TypeError when its argument is not a bytes object. On failure, the by-address output parameters readBuffer / readSize are not written. zstandard ignores the return code and proceeds to use those addresses, passing them to ZSTD_compressStream2 which reads from whatever happens to be on the stack / in registers.
Affected sites
| File | Line | Function |
|---|---|---|
c-ext/compressor.c |
349 | copy_stream |
c-ext/decompressor.c |
202 | decompressor_copy_stream |
c-ext/compressoriterator.c |
98 | ZstdCompressorIterator_iternext |
c-ext/decompressoriterator.c |
134 | ZstdDecompressorIterator_iternext |
Plus 2 additional sites reported in the full analysis (in read_compressor_input and the decompressor equivalent).
Suggested fix
Mechanical — add the standard error check after every call:
if (PyBytes_AsStringAndSize(result, &readBuffer, &readSize) < 0) {
Py_DECREF(result);
goto finally;
}
Optionally: tighten the documented API contract on source.read() to specify that the return must be a bytes object (the C code already expects this). Enforcing it at the boundary would be a small additional cleanup.
Methodology
Found via cext-review-toolkit (Tree-sitter-based static analysis with structured naive/informed review passes). SEGV on release verified at the copy_stream site; assertion abort on debug verified at the iterator site. Four sites confirmed via direct reproducer; two more confirmed via static review. Happy to open a PR — the fix is a ~8-line diff.
Discovery, root-cause analysis, and issue drafting were performed by Claude Code and reviewed by a human before filing.
Full report
Complete multi-agent analysis (48 FIX findings across 13 categories, plus a reproducer appendix): https://gist.github.com/devdanzin/b86039ac097141579590c1a0f3a43605
- 主要语言
- C
- 星标
- 642
- 派生
- 118
- 平均合并
- 9 天 3 小时
- 30 天内合并 PR
- 6
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
indygreg/python-zstandard 的其他 Issue
-
`multi_decompress_to_buffer([])` terminates the process with SIGFPE可能已有人在做 @mikamikasuki 于 17 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
indygreg/python-zstandard#335 ·
维护者通常 10 天内回复
-
Silent data-correctness bug: `readinto()` / `readinto1()` on `stream_reader` return `tell() == 0` after successful reads可能已有人在做 @CAOShurong 于 10 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
indygreg/python-zstandard#295 ·
维护者通常 10 天内回复
-
CFFI backend: crash when ZstdDecompressor instances share a ZstdCompressionDict可能已有人在做 @pylaterreur 于 2 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 45/100
indygreg/python-zstandard#361 ·
维护者通常 10 天内回复
-
Python 3.15 wheels可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 55/100
indygreg/python-zstandard#354 · 1 个 reaction ·
维护者通常 10 天内回复
-
`estimated_compression_context_size()` raises SIGFPE with `enable_ldm=True可能已有人在做 @yousefissa 于 4 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 64/100
indygreg/python-zstandard#345 ·
维护者通常 10 天内回复
查看 indygreg/python-zstandard 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
micropython/micropython#19786 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
vercel-labs/zerolang#442 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
维护者通常 1 天内回复
-
triage
难度 2/5 1-3 小时 新手友好度 67/100
维护者通常 1 天内回复
-
mewencode: base256[] and base128[] are indexed out of bounds by a non-ASCII octet可能已有人在做 @kazu-yamamoto 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
kazu-yamamoto/Mew#276 ·
维护者通常 1 天内回复