Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

[BUG] autobahn/__init__.py eagerly imports flatbuffers → numpy is loaded in every process (regression after 24.4.2)

Đang mở
  • Có gì không ổn?
#1,983 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
58/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
numpy, python
Lĩnh vực
backend, performance

Hướng nghiên cứu

Start by reading autobahn/__init__.py and trace its import of flatbuffers to flatbuffers/number_types.py, where NumPy is loaded at module scope. Compare that import path with the 24.4.2 behavior described in the issue, then check the existing tests around package imports and NumPy compatibility. Done means importing Autobahn without using FlatBuffers does not load NumPy, and a broken NumPy installation does not prevent importing Autobahn.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Summary

Since some release after 24.4.2, import autobahn imports the vendored flatbuffers
package at module-import time, and flatbuffers imports numpy whenever numpy
happens to be installed. Users who never touch WAMP or FlatBuffers now pay for numpy
in every process.

This reaches well beyond autobahn itself: daphne imports autobahn.twisted.websocket
from its Django AppConfig, so every Django + Channels project that has numpy
anywhere in its environment now loads numpy at django.setup() — web workers,
Celery workers, beat, management commands.

Import chain

daphne/apps.py:6                           import daphne.server
daphne/server.py:42
daphne/ws_protocol.py:6                    from autobahn.twisted.websocket import ...
autobahn/__init__.py:35                    from autobahn import flatbuffers
autobahn/flatbuffers/__init__.py:15        from . import util
autobahn/flatbuffers/util.py:15            from . import encode
autobahn/flatbuffers/encode.py:15          from . import number_types as N
autobahn/flatbuffers/number_types.py:21    np = import_numpy()
autobahn/flatbuffers/compat.py:75          import numpy as np

Two independent things combine here:

  1. autobahn/__init__.py gained an unconditional from autobahn import flatbuffers
    (it appears to serve check_zlmdb_flatbuffers_version_in_sync()). In 24.4.2 that
    module imported only _version, os and txaio.
  2. flatbuffers/number_types.py calls import_numpy() at module level, and
    compat.import_numpy() imports numpy whenever PathFinder.find_spec("numpy")
    succeeds — i.e. "numpy is installed" is treated as "numpy should be loaded".

Impact 1 — memory, for users who never use FlatBuffers

Measured with resource.getrusage(...).ru_maxrss on Python 3.13.13 / numpy 2.4.6
(macOS arm64):

process RSS
bare interpreter 16.4 MB
import txaio (what 24.4.2 pulled in) 16.3 MB
import numpy 29.4 MB

That is roughly +13 MB per process, multiplied by however many workers a
deployment runs. For a Django deployment with web + Celery + beat + a few
background workers this is a noticeable, invisible regression: nothing in the
application asks for numpy.

Impact 2 — a broken numpy now prevents importing autobahn at all

import_numpy() imports numpy deliberately outside try/except, with the comment
"We do this outside of try/except block in case numpy exists but is not installed
correctly". Combined with the eager import in __init__.py, a numpy whose import
fails — for example an ABI mismatch of the kind described in #1890 — makes
autobahn itself unimportable, and with it daphne and the whole Django app.

Reproducer for that failure mode:

import sys

class Block:
    def find_spec(self, name, path=None, target=None):
        if name == "numpy":
            raise ImportError("simulating a broken numpy install")
        return None

sys.meta_path.insert(0, Block())
import autobahn.twisted.websocket   # ImportError, raised from autobahn/__init__.py:35

Reproducer showing the import chain itself:

import sys, traceback

class Hook:
    def find_spec(self, name, path=None, target=None):
        if name == "numpy":
            traceback.print_stack()
            sys.meta_path.remove(self)
        return None

sys.meta_path.insert(0, Hook())
import autobahn.twisted.websocket
print("numpy" in sys.modules)   # True

Suggested fixes

Any one of these removes the cost for users who do not use FlatBuffers:

  1. Move from autobahn import flatbuffers inside
    check_zlmdb_flatbuffers_version_in_sync() (or behind a lazy accessor), so that
    importing autobahn does not import the serializer.
  2. Resolve numpy lazily in number_types.py instead of at module-import time.
  3. At minimum, wrap the numpy import in try/except ImportError, so a broken numpy
    degrades to "FlatBuffers numpy features unavailable" instead of
    "autobahn cannot be imported".

Options 1 and 2 are independent and either alone fixes the memory issue; 3 addresses
the robustness issue.

Environment

autobahn 26.7.1, daphne 4.2.2, numpy 2.4.6, Python 3.13.13, macOS arm64.

Noticed while upgrading autobahn to 26.7.1 for CVE-2026-77528: a guard test asserting
that numpy is absent from the Django setup import graph started failing.

Ngôn ngữ chính
Python
Star
2.5k
Fork
769
Merge trung bình
1 ngày 12 giờ
Pull request đã merge (30 ngày)
15

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của crossbario/autobahn-python

Tất cả issue của crossbario/autobahn-python

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.