Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[BUG] autobahn/__init__.py eagerly imports flatbuffers → numpy is loaded in every process (regression after 24.4.2)

Aperta
  • Qualcosa non va?
#1,983 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
58/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
numpy, python
Ambito
backend, performance

Direzione di ricerca

Start by reading autobahn/__init__.py and trace its import of flatbuffers to flatbuffers/number_types.py, where NumPy is loaded at module scope. Compare that import path with the 24.4.2 behavior described in the issue, then check the existing tests around package imports and NumPy compatibility. Done means importing Autobahn without using FlatBuffers does not load NumPy, and a broken NumPy installation does not prevent importing Autobahn.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Since some release after 24.4.2, import autobahn imports the vendored flatbuffers
package at module-import time, and flatbuffers imports numpy whenever numpy
happens to be installed. Users who never touch WAMP or FlatBuffers now pay for numpy
in every process.

This reaches well beyond autobahn itself: daphne imports autobahn.twisted.websocket
from its Django AppConfig, so every Django + Channels project that has numpy
anywhere in its environment now loads numpy at django.setup() — web workers,
Celery workers, beat, management commands.

Import chain

daphne/apps.py:6                           import daphne.server
daphne/server.py:42
daphne/ws_protocol.py:6                    from autobahn.twisted.websocket import ...
autobahn/__init__.py:35                    from autobahn import flatbuffers
autobahn/flatbuffers/__init__.py:15        from . import util
autobahn/flatbuffers/util.py:15            from . import encode
autobahn/flatbuffers/encode.py:15          from . import number_types as N
autobahn/flatbuffers/number_types.py:21    np = import_numpy()
autobahn/flatbuffers/compat.py:75          import numpy as np

Two independent things combine here:

  1. autobahn/__init__.py gained an unconditional from autobahn import flatbuffers
    (it appears to serve check_zlmdb_flatbuffers_version_in_sync()). In 24.4.2 that
    module imported only _version, os and txaio.
  2. flatbuffers/number_types.py calls import_numpy() at module level, and
    compat.import_numpy() imports numpy whenever PathFinder.find_spec("numpy")
    succeeds — i.e. "numpy is installed" is treated as "numpy should be loaded".

Impact 1 — memory, for users who never use FlatBuffers

Measured with resource.getrusage(...).ru_maxrss on Python 3.13.13 / numpy 2.4.6
(macOS arm64):

process RSS
bare interpreter 16.4 MB
import txaio (what 24.4.2 pulled in) 16.3 MB
import numpy 29.4 MB

That is roughly +13 MB per process, multiplied by however many workers a
deployment runs. For a Django deployment with web + Celery + beat + a few
background workers this is a noticeable, invisible regression: nothing in the
application asks for numpy.

Impact 2 — a broken numpy now prevents importing autobahn at all

import_numpy() imports numpy deliberately outside try/except, with the comment
"We do this outside of try/except block in case numpy exists but is not installed
correctly". Combined with the eager import in __init__.py, a numpy whose import
fails — for example an ABI mismatch of the kind described in #1890 — makes
autobahn itself unimportable, and with it daphne and the whole Django app.

Reproducer for that failure mode:

import sys

class Block:
    def find_spec(self, name, path=None, target=None):
        if name == "numpy":
            raise ImportError("simulating a broken numpy install")
        return None

sys.meta_path.insert(0, Block())
import autobahn.twisted.websocket   # ImportError, raised from autobahn/__init__.py:35

Reproducer showing the import chain itself:

import sys, traceback

class Hook:
    def find_spec(self, name, path=None, target=None):
        if name == "numpy":
            traceback.print_stack()
            sys.meta_path.remove(self)
        return None

sys.meta_path.insert(0, Hook())
import autobahn.twisted.websocket
print("numpy" in sys.modules)   # True

Suggested fixes

Any one of these removes the cost for users who do not use FlatBuffers:

  1. Move from autobahn import flatbuffers inside
    check_zlmdb_flatbuffers_version_in_sync() (or behind a lazy accessor), so that
    importing autobahn does not import the serializer.
  2. Resolve numpy lazily in number_types.py instead of at module-import time.
  3. At minimum, wrap the numpy import in try/except ImportError, so a broken numpy
    degrades to "FlatBuffers numpy features unavailable" instead of
    "autobahn cannot be imported".

Options 1 and 2 are independent and either alone fixes the memory issue; 3 addresses
the robustness issue.

Environment

autobahn 26.7.1, daphne 4.2.2, numpy 2.4.6, Python 3.13.13, macOS arm64.

Noticed while upgrading autobahn to 26.7.1 for CVE-2026-77528: a guard test asserting
that numpy is absent from the Django setup import graph started failing.

Lingua principale
Python
Stelle
2.5k
Fork
769
Merge medio
1g 12h
PR unite (30g)
15

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di crossbario/autobahn-python

Tutte le issue di crossbario/autobahn-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.