Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Segfault: 12 types dereference NULL on `Type.__new__(Type)` without `__init__()`

Abierto
  • ¿Algo no está bien?
#291 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Tranquilo
Stack tecnológico
c, python
Área
api, backend

Línea de trabajo

Comienza con las definiciones de PyType_Slot y las rutas de inicialización en los archivos c-ext afectados, especialmente compressor.c, decompressor.c, compressionparams.c, compressionwriter.c, decompressionwriter.c, compressionreader.c, bufferutil.c y los archivos de factory-type. Ejecuta los reproductores directos de una línea y los reproductores de una línea con type() indicados antes y después del cambio. Se considera terminado cuando los 13 tipos afectados gestionen de forma segura la construcción sin init, mientras la inicialización y la limpieza normales sigan siendo correctas.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

PyType_GenericNew creates zero-initialized instances, but zstandard's method implementations assume the C-level pointer fields (cctx, dctx, params, ...) are non-NULL. T.__new__(T).method(...) therefore segfaults for 12 of 13 extension types with nothing more than a standard-library import.

Impact

  • Severity: Segfault (some paths hit an assertion abort on debug builds).
  • Reachability: A single line of pure Python on a public type. No ZSTD-library involvement, no unusual input.
  • Version: 0.25.0 (commit 7a77a75); pattern very likely present in prior releases as well.
  • Platform: Confirmed Linux x86_64 / CPython 3.14 debug; the bug is platform-independent.

Reproducers

Each one-liner segfaults:

import zstandard as zstd

zstd.ZstdCompressor.__new__(zstd.ZstdCompressor).compress(b'x')
zstd.ZstdDecompressor.__new__(zstd.ZstdDecompressor).decompress(b'x')
zstd.ZstdCompressionParameters.__new__(zstd.ZstdCompressionParameters).estimated_compression_context_size()
zstd.ZstdCompressionWriter.__new__(zstd.ZstdCompressionWriter).write(b'x')
zstd.ZstdDecompressionWriter.__new__(zstd.ZstdDecompressionWriter).write(b'x')
zstd.ZstdCompressionReader.__new__(zstd.ZstdCompressionReader).read(10)
zstd.BufferWithSegmentsCollection.__new__(zstd.BufferWithSegmentsCollection)[0]

Five more types aren't in the top-level namespace but are reachable via type() introspection and crash the same way:

c = zstd.ZstdCompressor()
T = type(c.compressobj())        # ZstdCompressionObj
T.__new__(T).compress(b'x')      # segfault
# likewise: ZstdDecompressionObj, ZstdCompressorIterator,
#           ZstdDecompressorIterator, ZstdCompressionChunker

ZstdDecompressionReader (the 13th affected type) does not crash — read() returns b'' because input.pos == input.size == 0 takes an early-return branch. The instance is still in an invalid state; any future method that doesn't short-circuit on this state will crash.

Root cause

All 13 type specs install {Py_tp_new, PyType_GenericNew}. PyType_GenericNew zero-initializes the instance. tp_init is where real allocation (ZSTD_createCCtx, ZSTD_createDCtx, PyMem_Malloc, ...) happens — skip __init__ and the pointers stay NULL. Methods then do things like ZSTD_CCtx_reset(self->cctx, ...) on NULL.

Affected types

Type File NULL field / observable Trigger
ZstdCompressor c-ext/compressor.c cctx .compress(b'x')
ZstdDecompressor c-ext/decompressor.c dctx (via ensure_dctx) .decompress(b'x')
ZstdCompressionParameters c-ext/compressionparams.c params any accessor
ZstdCompressionWriter c-ext/compressionwriter.c compressor .write(b'x')
ZstdDecompressionWriter c-ext/decompressionwriter.c decompressor .write(b'x')
ZstdCompressionReader c-ext/compressionreader.c assertion abort .read(10)
BufferWithSegmentsCollection c-ext/bufferutil.c firstElements [0]
ZstdCompressionObj c-ext/compressobj.c compressor via type()
ZstdDecompressionObj c-ext/decompressobj.c decompressor via type()
ZstdCompressorIterator c-ext/compressoriterator.c compressor/reader via type()
ZstdDecompressorIterator c-ext/decompressoriterator.c decompressor/reader via type()
ZstdCompressionChunker c-ext/compressionchunker.c compressor via type()
ZstdDecompressionReader (silent) c-ext/decompressionreader.c zero-state early return .read(10) → b''

Suggested fix

Two options; Option A is recommended. It yields a usable object immediately after __new__ and composes cleanly with a re-init leak finding noted below.

Option A — allocate in tp_new

Replace {Py_tp_new, PyType_GenericNew} with a type-specific tp_new that installs the minimum non-NULL state the methods need:

static PyObject *
ZstdCompressor_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    ZstdCompressor *self = (ZstdCompressor *)type->tp_alloc(type, 0);
    if (!self) {
        return NULL;
    }
    self->cctx = ZSTD_createCCtx();
    if (!self->cctx) {
        Py_DECREF(self);
        PyErr_NoMemory();
        return NULL;
    }
    /* other fields stay zero-initialized; tp_init will configure them. */
    return (PyObject *)self;
}

static PyType_Slot ZstdCompressor_slots[] = {
    {Py_tp_dealloc, ZstdCompressor_dealloc},
    {Py_tp_methods, ZstdCompressor_methods},
    {Py_tp_init,    (initproc)ZstdCompressor_init},
    {Py_tp_new,     ZstdCompressor_new},      /* was: PyType_GenericNew */
    {0, 0},
};

tp_init then only configures the already-allocated context. For types whose tp_init also allocates (ZstdCompressionParameters, BufferWithSegmentsCollection, ...), the allocation moves into tp_new and tp_init is reduced to argument parsing + configuration.

Option B — guard every method

Add a NULL check at the top of every method that dereferences an initializable field:

static PyObject *
ZstdCompressor_compress(ZstdCompressor *self, PyObject *args)
{
    if (!self->cctx) {
        PyErr_SetString(PyExc_ValueError,
                        "ZstdCompressor not initialized - call __init__");
        return NULL;
    }
    /* ... */
}

Less invasive but requires auditing every method on every affected type, and new methods have to remember the guard.

Related / follow-ups

  • This composes with a re-init leak finding (comp.__init__(...) on an already-initialized instance allocates new contexts without freeing the previous ones — ~5.5 KB per call) covered in the full report linked below. Option A cleanly separates "create fresh state" (tp_new) from "configure from kwargs" (tp_init), making it easy to either reject re-init or free-then-recreate.
  • The ZstdCompressionObj / ZstdDecompressionObj / iterator / chunker types are returned from factories and not directly constructable, but their __new__ remains reachable via type(). Same fix applies.

Methodology

Found via cext-review-toolkit (Tree-sitter-based static analysis with structured naive/informed review passes). All 7 direct-namespace reproducers and 5 type()-introspection reproducers were verified live on CPython 3.14.3 debug build. Happy to open a PR — the Option A change is mechanical across the 13 specs. I'd propose a single PR with the atomic set of changes, but can split into a 2-commit PR (simple-allocation types first, more-complex-init types second) if you prefer.

Discovery, root-cause analysis, and issue drafting were performed by Claude Code and reviewed by a human before filing.

Full report

Complete multi-agent analysis (48 FIX findings across 13 categories, plus a reproducer appendix): https://gist.github.com/devdanzin/b86039ac097141579590c1a0f3a43605

Lenguaje dominante
C
Estrellas
641
Forks
117
Merge medio
1 d 14 h
PR fusionados (30 d)
5

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de indygreg/python-zstandard

Todos los issues de indygreg/python-zstandard

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.