Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Python 3.14: crash or hang at exit while daemon threads scan object/Arrow data (not fixed by #709)

Đang mở
  • Có gì không ổn?
#714 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
38/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
pandas, python
Lĩnh vực
backend, databases

Hướng nghiên cứu

Start with the two failing cases in tests/slow/test_interpreter_shutdown.py and compare the object and Arrow scan paths with the numeric and string[pyarrow] cases that pass. Read the related issues #601, #638, #531, and #709, then investigate how scan callbacks acquire the GIL during Python 3.14 finalization. Done means both cases exit reliably without crashes or hangs on 3.14 while the unaffected cases continue to pass.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

What happens?

On Python 3.14, a process whose daemon threads are still running queries that scan Python objects (an object-dtype pandas column, or a pyarrow table) crashes or hangs at interpreter exit. The same scripts exit cleanly on Python 3.13.

These are the daemon_threads_scanning_object_frame_at_exit and daemon_threads_scanning_one_chunk_arrow_table_at_exit cases in tests/slow/test_interpreter_shutdown.py. They fail intermittently on 3.14. Daemon threads that scan numeric or string[pyarrow] frames, or that only run range() queries, never fail.

#709 (end every open result before the interpreter finalizes) helps the object-frame case (37 → 11 failures per 100) but not the Arrow-table case (7 → 9), so this looks like a separate path: DuckDB's worker threads still enter the Python C API during 3.14 finalization, while the daemon threads' queries are running.

Failure signatures (a mix, from run to run):

  • SIGSEGV, exit status -11, usually right after nanobind's leak report
  • double free or corruption (out) followed by Fatal Python error: PyInterpreterState_Delete: remaining threads (Python runtime state: finalizing)
  • Fatal Python error: PyGILState_Release: auto-releasing thread-state, but no thread-state for this thread
  • Fatal Python error: gilstate_tss_set: failed to set current tstate (TSS)
  • On Windows (CI): exit status 0xC0000409 (STATUS_STACK_BUFFER_OVERRUN)
  • A hang at exit: the process never terminates and has to be killed

Failures per 100 runs of each test case, using the test's own pass rule. Linux aarch64, 12 runs in parallel:

build Python object_frame one_chunk_arrow_table
duckdb==2.0.0 (PyPI wheel) 3.14.7 37 7
duckdb==2.0.0 (PyPI wheel) 3.13.15 0 0
downstream build of v2.0.0 with #709 cherry-picked (and its test updates) 3.14.7 11 9
same 3.13.15 0 0

With 3 runs in parallel, duckdb==2.0.0 on 3.14.7 failed 19 (object_frame) and 17 (one_chunk_arrow_table) out of 100. We have also seen both cases fail on macOS arm64 and Windows amd64 CI runners with the cp312-abi3 wheel on 3.14.

Possibly related: #601 (don't touch Python state from destructors that outlive the interpreter), #638 and #531.

One possible direction: before a scan callback takes the GIL from a DuckDB worker thread, check Py_IsFinalizing() (or _Py_IsFinalizing()). If finalization has started, fail the task with an interrupt instead of calling PyGILState_Ensure. Together with #709's atexit hook, that would cover a query that starts or resumes after the hook has run.

To Reproduce
import threading
import time

import duckdb
import pandas as pd

frame = pd.DataFrame({"a": pd.Series([f"v{i % 100}" for i in range(200_000)], dtype=object)})
con = duckdb.connect()


def work():
    cursor = con.cursor()
    while True:
        cursor.execute("select count(a), min(a) from frame").fetchall()


for _ in range(4):
    threading.Thread(target=work, daemon=True).start()
time.sleep(0.5)
print("done", flush=True)

Run it repeatedly on Python 3.14 with a timeout, because some runs hang:

for i in $(seq 1 100); do timeout 20 python repro.py >/dev/null 2>err.txt; echo $?; done | sort | uniq -c

With duckdb==2.0.0 from PyPI on Linux aarch64, one run at a time, the script above hung at exit in 4 of 100 runs on Python 3.14.7 and in 0 of 50 on Python 3.13.15. With 12 runs in parallel, as in the table above, the crashes appear as well.

OS:

Amazon Linux 2023 (aarch64); also seen on macOS arm64 and Windows amd64 GitHub Actions runners

DuckDB Package Version:

2.0.0

Python Version:

3.14.7 (3.13.15 is not affected)

Full Name:

Rusty Conover

Affiliation:

Query.Farm

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I tested with the latest release.

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?
  • Yes, I have
Did you include all relevant configuration to reproduce the issue?
  • Yes, I have

🤖 Generated with Claude Code

Ngôn ngữ chính
Python
Star
194
Fork
119
Merge trung bình
19 giờ 31 phút
Pull request đã merge (30 ngày)
50

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của duckdb/duckdb-python

Tất cả issue của duckdb/duckdb-python

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.