Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Segfault: 12 types dereference NULL on `Type.__new__(Type)` without `__init__()`

Aperta
  • Qualcosa non va?
#291 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Tranquilla
Stack tecnologico
c, python
Ambito
api, backend

Direzione di ricerca

Inizia dalle definizioni di PyType_Slot e dai percorsi di inizializzazione nei file c-ext interessati, in particolare compressor.c, decompressor.c, compressionparams.c, compressionwriter.c, decompressionwriter.c, compressionreader.c, bufferutil.c e nei file dei factory-type. Esegui i riproduttori diretti su una riga e i riproduttori su una riga con type() elencati prima e dopo la modifica. Il lavoro è completato quando tutti i 13 tipi interessati gestiscono in sicurezza la costruzione senza init, mentre l’inizializzazione e la pulizia normali continuano a funzionare correttamente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

PyType_GenericNew creates zero-initialized instances, but zstandard's method implementations assume the C-level pointer fields (cctx, dctx, params, ...) are non-NULL. T.__new__(T).method(...) therefore segfaults for 12 of 13 extension types with nothing more than a standard-library import.

Impact

  • Severity: Segfault (some paths hit an assertion abort on debug builds).
  • Reachability: A single line of pure Python on a public type. No ZSTD-library involvement, no unusual input.
  • Version: 0.25.0 (commit 7a77a75); pattern very likely present in prior releases as well.
  • Platform: Confirmed Linux x86_64 / CPython 3.14 debug; the bug is platform-independent.

Reproducers

Each one-liner segfaults:

import zstandard as zstd

zstd.ZstdCompressor.__new__(zstd.ZstdCompressor).compress(b'x')
zstd.ZstdDecompressor.__new__(zstd.ZstdDecompressor).decompress(b'x')
zstd.ZstdCompressionParameters.__new__(zstd.ZstdCompressionParameters).estimated_compression_context_size()
zstd.ZstdCompressionWriter.__new__(zstd.ZstdCompressionWriter).write(b'x')
zstd.ZstdDecompressionWriter.__new__(zstd.ZstdDecompressionWriter).write(b'x')
zstd.ZstdCompressionReader.__new__(zstd.ZstdCompressionReader).read(10)
zstd.BufferWithSegmentsCollection.__new__(zstd.BufferWithSegmentsCollection)[0]

Five more types aren't in the top-level namespace but are reachable via type() introspection and crash the same way:

c = zstd.ZstdCompressor()
T = type(c.compressobj())        # ZstdCompressionObj
T.__new__(T).compress(b'x')      # segfault
# likewise: ZstdDecompressionObj, ZstdCompressorIterator,
#           ZstdDecompressorIterator, ZstdCompressionChunker

ZstdDecompressionReader (the 13th affected type) does not crash — read() returns b'' because input.pos == input.size == 0 takes an early-return branch. The instance is still in an invalid state; any future method that doesn't short-circuit on this state will crash.

Root cause

All 13 type specs install {Py_tp_new, PyType_GenericNew}. PyType_GenericNew zero-initializes the instance. tp_init is where real allocation (ZSTD_createCCtx, ZSTD_createDCtx, PyMem_Malloc, ...) happens — skip __init__ and the pointers stay NULL. Methods then do things like ZSTD_CCtx_reset(self->cctx, ...) on NULL.

Affected types

Type File NULL field / observable Trigger
ZstdCompressor c-ext/compressor.c cctx .compress(b'x')
ZstdDecompressor c-ext/decompressor.c dctx (via ensure_dctx) .decompress(b'x')
ZstdCompressionParameters c-ext/compressionparams.c params any accessor
ZstdCompressionWriter c-ext/compressionwriter.c compressor .write(b'x')
ZstdDecompressionWriter c-ext/decompressionwriter.c decompressor .write(b'x')
ZstdCompressionReader c-ext/compressionreader.c assertion abort .read(10)
BufferWithSegmentsCollection c-ext/bufferutil.c firstElements [0]
ZstdCompressionObj c-ext/compressobj.c compressor via type()
ZstdDecompressionObj c-ext/decompressobj.c decompressor via type()
ZstdCompressorIterator c-ext/compressoriterator.c compressor/reader via type()
ZstdDecompressorIterator c-ext/decompressoriterator.c decompressor/reader via type()
ZstdCompressionChunker c-ext/compressionchunker.c compressor via type()
ZstdDecompressionReader (silent) c-ext/decompressionreader.c zero-state early return .read(10) → b''

Suggested fix

Two options; Option A is recommended. It yields a usable object immediately after __new__ and composes cleanly with a re-init leak finding noted below.

Option A — allocate in tp_new

Replace {Py_tp_new, PyType_GenericNew} with a type-specific tp_new that installs the minimum non-NULL state the methods need:

static PyObject *
ZstdCompressor_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    ZstdCompressor *self = (ZstdCompressor *)type->tp_alloc(type, 0);
    if (!self) {
        return NULL;
    }
    self->cctx = ZSTD_createCCtx();
    if (!self->cctx) {
        Py_DECREF(self);
        PyErr_NoMemory();
        return NULL;
    }
    /* other fields stay zero-initialized; tp_init will configure them. */
    return (PyObject *)self;
}

static PyType_Slot ZstdCompressor_slots[] = {
    {Py_tp_dealloc, ZstdCompressor_dealloc},
    {Py_tp_methods, ZstdCompressor_methods},
    {Py_tp_init,    (initproc)ZstdCompressor_init},
    {Py_tp_new,     ZstdCompressor_new},      /* was: PyType_GenericNew */
    {0, 0},
};

tp_init then only configures the already-allocated context. For types whose tp_init also allocates (ZstdCompressionParameters, BufferWithSegmentsCollection, ...), the allocation moves into tp_new and tp_init is reduced to argument parsing + configuration.

Option B — guard every method

Add a NULL check at the top of every method that dereferences an initializable field:

static PyObject *
ZstdCompressor_compress(ZstdCompressor *self, PyObject *args)
{
    if (!self->cctx) {
        PyErr_SetString(PyExc_ValueError,
                        "ZstdCompressor not initialized - call __init__");
        return NULL;
    }
    /* ... */
}

Less invasive but requires auditing every method on every affected type, and new methods have to remember the guard.

Related / follow-ups

  • This composes with a re-init leak finding (comp.__init__(...) on an already-initialized instance allocates new contexts without freeing the previous ones — ~5.5 KB per call) covered in the full report linked below. Option A cleanly separates "create fresh state" (tp_new) from "configure from kwargs" (tp_init), making it easy to either reject re-init or free-then-recreate.
  • The ZstdCompressionObj / ZstdDecompressionObj / iterator / chunker types are returned from factories and not directly constructable, but their __new__ remains reachable via type(). Same fix applies.

Methodology

Found via cext-review-toolkit (Tree-sitter-based static analysis with structured naive/informed review passes). All 7 direct-namespace reproducers and 5 type()-introspection reproducers were verified live on CPython 3.14.3 debug build. Happy to open a PR — the Option A change is mechanical across the 13 specs. I'd propose a single PR with the atomic set of changes, but can split into a 2-commit PR (simple-allocation types first, more-complex-init types second) if you prefer.

Discovery, root-cause analysis, and issue drafting were performed by Claude Code and reviewed by a human before filing.

Full report

Complete multi-agent analysis (48 FIX findings across 13 categories, plus a reproducer appendix): https://gist.github.com/devdanzin/b86039ac097141579590c1a0f3a43605

Lingua principale
C
Stelle
641
Fork
117
Merge medio
1g 14h
PR unite (30g)
5

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di indygreg/python-zstandard

Tutte le issue di indygreg/python-zstandard

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.