Thin mode asyncio: segfault in data_received after a late FIN from a redirecting listener
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 15/100
- Loại issue
- Lỗi
- Độ rõ ràng
- Khá rõ ràng
- Mức độ hoạt động
- Sôi nổi
- Công nghệ
- python
- Lĩnh vực
- database, networking
Hướng nghiên cứu
Start in src/oracledb/impl/thin/protocol.pyx: read _connect_tcp, where a redirect opens a second transport on the same protocol, then connection_lost and data_received, where self._transport can be None. Check _post_connect in connection.pyx, which clears _in_connect. Done means a late FIN after a redirect no longer nulls the live transport, data_received does not dereference None, and the reproduction script survives with FIN_DELAY = 1.0.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
This looks like the same root cause as #616, but it ends in a crash instead of an error, and the fix in #617 would not stop it. That's why I'm opening it separately. Happy to move it to #616 if you prefer.
1. What versions are you using?
- python-oracledb 26.1.0 and 3.4.2, both crash with the script below. 3.4.1 crashed the same way when I tried it with a fake redirecting listener.
- Oracle AI Database 26ai Free 23.26.3.0.0 (
gvenzl/oracle-free:23-slim-faststart) - Python 3.14.10 on Linux aarch64, in Docker. Same crash on Linux x86_64, where I used 26.1.0 and 3.4.1 with the fake listener and 3.4.2 with the real one.
2. Is it an error or a hang or a crash?
A crash. SIGSEGV, exit code 139, the whole process goes down.
3. What error(s) or behavior you are seeing?
We noticed this in a long-running asyncio service using Thin mode pools. It died with exit 139 a few times a day, always at the same place in thin_impl. Our listeners appear to redirect too. I have no core dump from there, so I can't prove a late FIN is what happened in that service, but the script below crashes at the same Python frame and, on 3.4.2 x86_64, at the same offset in thin_impl.
As far as I can tell this is what happens:
- On a redirect,
_connect_tcpcallscreate_connection(lambda: self, ...)again and the first transport stays open, so two transports share one protocol. connection_lost()doesn't know which transport was lost. It only ignores the loss while_in_connectis set, and_post_connectclears that flag.- If the listener's FIN arrives after that point,
self._transportis set toNone, even though the second transport is healthy and still reading. - The running call fails with DPY-4011. When the server's reply then arrives on the live socket,
data_received()callsself._transport.extract_packet(data)onNoneand the process segfaults. On 3.4.2 x86_64 I mapped the crash address to exactly that call.
Normally the listener's FIN arrives a few milliseconds after the redirect and wins the race, so you need a FIN that shows up late, for example from a stalled listener or a retransmission. In #616 the listener closes about 0.3 s later on a 19c RAC, so this doesn't seem that exotic.
Output from 26.1.0 (trimmed, stdout was lost in the crash):
Fatal Python error: Segmentation fault
Current thread 0x0000eb8a59ee0020 [python] (most recent call first):
File "/usr/local/lib/python3.14/asyncio/selector_events.py", line 1025 in _read_ready__data_received
File "/usr/local/lib/python3.14/asyncio/selector_events.py", line 968 in _read_ready
File "/usr/local/lib/python3.14/asyncio/events.py", line 94 in _run
File "/usr/local/lib/python3.14/asyncio/base_events.py", line 2064 in _run_once
...
Current thread's C stack trace (most recent call first):
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at _Py_DumpStack+0x58 [0xeb8a599c6218]
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at +0xdfcfc [0xeb8a599cfcfc]
Binary file "linux-vdso.so.1", at __kernel_rt_sigreturn+0x0 [0xeb8a59ee88f8]
Binary file "/usr/local/lib/python3.14/site-packages/oracledb/thin_impl.cpython-314-aarch64-linux-gnu.so", at +0x1f80dc [0xeb8a57ec80dc]
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x54 [0xeb8a59a06a64]
...
With FIN_DELAY = 0 the same script is fine:
oracledb 26.1.0 python 3.14.10
connected 23.26.3.0.0
waiting for the server's reply on the live socket
survived
Two more things I saw on 3.4.2 from the same cause:
- If no call is in flight when the late FIN arrives, nothing crashes, but the connection reports DPY-4011 although its socket is fine.
close()then does nothing because_transportisNone, so the socket stays ESTABLISHED after close. Pools replace these connections, so you only notice the leaked sockets. - If the FIN arrives while connect is waiting for a packet, the connect fails. That's #616.
4. Does your application call init_oracle_client()?
No, Thin mode.
5. Include a runnable Python script that shows the problem.
It needs a listener that really redirects, so I used shared server with direct handoff off. As SYSDBA:
alter system set shared_servers=1 scope=memory;
alter system set dispatchers='(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=15299))(DISPATCHERS=1)' scope=memory;
alter system register;
Then add this to listener.ora and run lsnrctl reload:
DIRECT_HANDOFF_TTC_LISTENER=OFF
The user is probe / probe in FREEPDB1. Run the script with the client in the database container's network namespace (docker run --network container:<name> ...), so it can reach the dispatcher on 127.0.0.1.
"""Thin mode, asyncio: late FIN from the redirecting listener, then SIGSEGV.
The script puts a small TCP proxy in front of the listener. Data passes through
untouched; the only change is that the listener's FIN reaches the client one
second late. Run it on the database host, because the client must be able to
reach the dispatcher address the listener redirects to.
"""
import asyncio
import faulthandler
import sys
import threading
import time
import oracledb
LISTENER = ("127.0.0.1", 1521)
PROXY_PORT = 15213
FIN_DELAY = 1.0
USER = "probe"
PASSWORD = "probe"
SERVICE = "FREEPDB1"
async def copy(reader, writer):
while data := await reader.read(65536):
writer.write(data)
await writer.drain()
async def handle(client_reader, client_writer):
server_reader, server_writer = await asyncio.open_connection(*LISTENER)
upstream = asyncio.create_task(copy(client_reader, server_writer))
await copy(server_reader, client_writer)
await asyncio.sleep(FIN_DELAY)
client_writer.close()
server_writer.close()
upstream.cancel()
async def proxy():
server = await asyncio.start_server(handle, "127.0.0.1", PROXY_PORT)
await server.serve_forever()
async def client():
conn = await oracledb.connect_async(
user=USER,
password=PASSWORD,
host="127.0.0.1",
port=PROXY_PORT,
service_name=SERVICE,
server_type="shared",
)
print("connected", conn.version)
cursor = conn.cursor()
try:
await cursor.execute("begin dbms_session.sleep(3); end;")
except oracledb.Error as error:
print("query raised:", error)
print("waiting for the server's reply on the live socket")
await asyncio.sleep(6)
print("survived")
if __name__ == "__main__":
faulthandler.enable()
print("oracledb", oracledb.__version__, "python", sys.version.split()[0])
threading.Thread(target=lambda: asyncio.run(proxy()), daemon=True).start()
time.sleep(1)
asyncio.run(client())
26.1.0 and 3.4.2 each crashed 2 out of 2 runs. The control with FIN_DELAY = 0 ran once on 26.1.0 and survived.
Possible fix
#617 won't help here, because its check only runs while _in_connect is set and this happens after that.
I think the cleanest fix is to close the previous transport in _connect_tcp before opening the new one, so a late FIN can never reach the protocol. Guarding both extract_packet calls in data_received against self._transport is None also seems worth doing, since a NULL dispatch shouldn't be able to take the interpreter down.
As a workaround we wrap loop.create_connection so it closes the previous transport when the same protocol comes back. That removed the crash and the DPY-4011 in my tests. Our service hasn't crashed in the day and a half since, but that's early days.
Disclaimer: this issue was generated by AI. The script and the output above come from real runs against a local Oracle Free database.
- Ngôn ngữ chính
- Python
- Star
- 456
- Fork
- 119
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của oracle/python-oracledb
-
bug
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 22/100
oracle/python-oracledb#628 ·
-
Thin mode: ORA-01483 for string binds of 2731-8191 bytes on a database with the UTF8 character setĐang mở
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 45/100
oracle/python-oracledb#627 · 1 bình luận ·
-
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 75/100
oracle/python-oracledb#626 ·
-
bug
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 48/100
oracle/python-oracledb#625 ·
-
bug
Độ khó 4/5 1-2 ngày Mức phù hợp với người mới 45/100
oracle/python-oracledb#624 ·
Tất cả issue của oracle/python-oracledb
Issue tương tự
-
Paraxial.trace returns NaN for an object at infinity when the entrance pupil is at the first surfaceĐang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 75/100
Maintainer thường phản hồi trong vòng 1 ngày
-
enhancement ios launcher
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 70/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Substrait consumer doesn't recognise `shift_left` / `shift_right` from functions_arithmetic.yamlĐang mởbug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
apache/datafusion#27059 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 85/100
loteran/Arctis-Sound-Manager#368 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
feed submission
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 65/100
MobilityData/mobility-database-catalogs#1787 ·
Maintainer thường phản hồi trong vòng 2 ngày