Thin mode asyncio: segfault in data_received after a late FIN from a redirecting listener
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 15/100
- Type d'issue
- Bug
- Clarté
- Plutôt claire
- Activité
- Active
- Stack technique
- python
- Domaine
- database, networking
Piste de recherche
Start in src/oracledb/impl/thin/protocol.pyx: read _connect_tcp, where a redirect opens a second transport on the same protocol, then connection_lost and data_received, where self._transport can be None. Check _post_connect in connection.pyx, which clears _in_connect. Done means a late FIN after a redirect no longer nulls the live transport, data_received does not dereference None, and the reproduction script survives with FIN_DELAY = 1.0.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
This looks like the same root cause as #616, but it ends in a crash instead of an error, and the fix in #617 would not stop it. That's why I'm opening it separately. Happy to move it to #616 if you prefer.
1. What versions are you using?
- python-oracledb 26.1.0 and 3.4.2, both crash with the script below. 3.4.1 crashed the same way when I tried it with a fake redirecting listener.
- Oracle AI Database 26ai Free 23.26.3.0.0 (
gvenzl/oracle-free:23-slim-faststart) - Python 3.14.10 on Linux aarch64, in Docker. Same crash on Linux x86_64, where I used 26.1.0 and 3.4.1 with the fake listener and 3.4.2 with the real one.
2. Is it an error or a hang or a crash?
A crash. SIGSEGV, exit code 139, the whole process goes down.
3. What error(s) or behavior you are seeing?
We noticed this in a long-running asyncio service using Thin mode pools. It died with exit 139 a few times a day, always at the same place in thin_impl. Our listeners appear to redirect too. I have no core dump from there, so I can't prove a late FIN is what happened in that service, but the script below crashes at the same Python frame and, on 3.4.2 x86_64, at the same offset in thin_impl.
As far as I can tell this is what happens:
- On a redirect,
_connect_tcpcallscreate_connection(lambda: self, ...)again and the first transport stays open, so two transports share one protocol. connection_lost()doesn't know which transport was lost. It only ignores the loss while_in_connectis set, and_post_connectclears that flag.- If the listener's FIN arrives after that point,
self._transportis set toNone, even though the second transport is healthy and still reading. - The running call fails with DPY-4011. When the server's reply then arrives on the live socket,
data_received()callsself._transport.extract_packet(data)onNoneand the process segfaults. On 3.4.2 x86_64 I mapped the crash address to exactly that call.
Normally the listener's FIN arrives a few milliseconds after the redirect and wins the race, so you need a FIN that shows up late, for example from a stalled listener or a retransmission. In #616 the listener closes about 0.3 s later on a 19c RAC, so this doesn't seem that exotic.
Output from 26.1.0 (trimmed, stdout was lost in the crash):
Fatal Python error: Segmentation fault
Current thread 0x0000eb8a59ee0020 [python] (most recent call first):
File "/usr/local/lib/python3.14/asyncio/selector_events.py", line 1025 in _read_ready__data_received
File "/usr/local/lib/python3.14/asyncio/selector_events.py", line 968 in _read_ready
File "/usr/local/lib/python3.14/asyncio/events.py", line 94 in _run
File "/usr/local/lib/python3.14/asyncio/base_events.py", line 2064 in _run_once
...
Current thread's C stack trace (most recent call first):
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at _Py_DumpStack+0x58 [0xeb8a599c6218]
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at +0xdfcfc [0xeb8a599cfcfc]
Binary file "linux-vdso.so.1", at __kernel_rt_sigreturn+0x0 [0xeb8a59ee88f8]
Binary file "/usr/local/lib/python3.14/site-packages/oracledb/thin_impl.cpython-314-aarch64-linux-gnu.so", at +0x1f80dc [0xeb8a57ec80dc]
Binary file "/usr/local/bin/../lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x54 [0xeb8a59a06a64]
...
With FIN_DELAY = 0 the same script is fine:
oracledb 26.1.0 python 3.14.10
connected 23.26.3.0.0
waiting for the server's reply on the live socket
survived
Two more things I saw on 3.4.2 from the same cause:
- If no call is in flight when the late FIN arrives, nothing crashes, but the connection reports DPY-4011 although its socket is fine.
close()then does nothing because_transportisNone, so the socket stays ESTABLISHED after close. Pools replace these connections, so you only notice the leaked sockets. - If the FIN arrives while connect is waiting for a packet, the connect fails. That's #616.
4. Does your application call init_oracle_client()?
No, Thin mode.
5. Include a runnable Python script that shows the problem.
It needs a listener that really redirects, so I used shared server with direct handoff off. As SYSDBA:
alter system set shared_servers=1 scope=memory;
alter system set dispatchers='(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=15299))(DISPATCHERS=1)' scope=memory;
alter system register;
Then add this to listener.ora and run lsnrctl reload:
DIRECT_HANDOFF_TTC_LISTENER=OFF
The user is probe / probe in FREEPDB1. Run the script with the client in the database container's network namespace (docker run --network container:<name> ...), so it can reach the dispatcher on 127.0.0.1.
"""Thin mode, asyncio: late FIN from the redirecting listener, then SIGSEGV.
The script puts a small TCP proxy in front of the listener. Data passes through
untouched; the only change is that the listener's FIN reaches the client one
second late. Run it on the database host, because the client must be able to
reach the dispatcher address the listener redirects to.
"""
import asyncio
import faulthandler
import sys
import threading
import time
import oracledb
LISTENER = ("127.0.0.1", 1521)
PROXY_PORT = 15213
FIN_DELAY = 1.0
USER = "probe"
PASSWORD = "probe"
SERVICE = "FREEPDB1"
async def copy(reader, writer):
while data := await reader.read(65536):
writer.write(data)
await writer.drain()
async def handle(client_reader, client_writer):
server_reader, server_writer = await asyncio.open_connection(*LISTENER)
upstream = asyncio.create_task(copy(client_reader, server_writer))
await copy(server_reader, client_writer)
await asyncio.sleep(FIN_DELAY)
client_writer.close()
server_writer.close()
upstream.cancel()
async def proxy():
server = await asyncio.start_server(handle, "127.0.0.1", PROXY_PORT)
await server.serve_forever()
async def client():
conn = await oracledb.connect_async(
user=USER,
password=PASSWORD,
host="127.0.0.1",
port=PROXY_PORT,
service_name=SERVICE,
server_type="shared",
)
print("connected", conn.version)
cursor = conn.cursor()
try:
await cursor.execute("begin dbms_session.sleep(3); end;")
except oracledb.Error as error:
print("query raised:", error)
print("waiting for the server's reply on the live socket")
await asyncio.sleep(6)
print("survived")
if __name__ == "__main__":
faulthandler.enable()
print("oracledb", oracledb.__version__, "python", sys.version.split()[0])
threading.Thread(target=lambda: asyncio.run(proxy()), daemon=True).start()
time.sleep(1)
asyncio.run(client())
26.1.0 and 3.4.2 each crashed 2 out of 2 runs. The control with FIN_DELAY = 0 ran once on 26.1.0 and survived.
Possible fix
#617 won't help here, because its check only runs while _in_connect is set and this happens after that.
I think the cleanest fix is to close the previous transport in _connect_tcp before opening the new one, so a late FIN can never reach the protocol. Guarding both extract_packet calls in data_received against self._transport is None also seems worth doing, since a NULL dispatch shouldn't be able to take the interpreter down.
As a workaround we wrap loop.create_connection so it closes the previous transport when the same protocol comes back. That removed the crash and the DPY-4011 in my tests. Our service hasn't crashed in the day and a half since, but that's early days.
Disclaimer: this issue was generated by AI. The script and the output above come from real runs against a local Oracle Free database.
- Langage dominant
- Python
- Étoiles
- 456
- Forks
- 119
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Préparer son environnement
- Aucun Dockerfile ni fichier Docker Compose
- Propose un modèle de pull request
- Lire le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de oracle/python-oracledb
-
Difficulté 4/5 3-5 jours Accessibilité débutants 48/100
oracle/python-oracledb#630 ·
-
bug
Difficulté 4/5 3-5 jours Accessibilité débutants 22/100
oracle/python-oracledb#628 ·
-
Thin mode: ORA-01483 for string binds of 2731-8191 bytes on a database with the UTF8 character setOuverte
Difficulté 4/5 3-5 jours Accessibilité débutants 45/100
oracle/python-oracledb#627 · 1 commentaire · 1 réaction ·
-
Difficulté 3/5 1-2 jours Accessibilité débutants 75/100
oracle/python-oracledb#626 ·
-
bug
Difficulté 4/5 3-5 jours Accessibilité débutants 48/100
oracle/python-oracledb#625 ·
Toutes les issues de oracle/python-oracledb
Issues similaires
-
flow-measurement measurement
Difficulté 1/5 Moins d'une heure Accessibilité débutants 76/100
MichaelYcJo/SpecSeal#1366 ·
-
bug c/evals c/ui size:L
Difficulté 2/5 1-3 heures Accessibilité débutants 66/100
Arize-ai/phoenix#17633 · 2 commentaires · 1 réaction ·
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
TransformerLensOrg/TransformerLens#1980 ·
Les mainteneurs répondent en général sous 1 jour
-
Module: Solr Needs: Lead Needs: Triage Theme: Search Type: Bug
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
internetarchive/openlibrary#14284 · 1 commentaire ·
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 1/5 Moins d'une heure Accessibilité débutants 69/100
SpaceCowMedia/commander-spellbook-backend#1292 · 1 commentaire ·
Les mainteneurs répondent en général sous 1 jour