Free-threading race betwen two Py_SETREF

Offen
  • Stimmt etwas nicht?
#161,714 3 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
22/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
c, python
Bereich
backend

Rechercherichtung

The race is in the Py_SETREF and Py_XSETREF macros, which copy the old pointer, store the new one, and then decref in separate non-atomic steps. Run the reproducer with a free-threaded debug build and read the assertion at Include/refcount.h:617 to confirm the negative refcount on a tuple. Done would mean the reproducer runs clean under free-threading, which needs a design decision about atomic swap semantics first.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

interpreter-core topic-free-threading type-bug
Bug report

This can be triggered with existing Python objects that use Py_SETREF or Py_XSETREF such as Exceptions (although slightly contrived):

import threading

e = Exception()

def worker():
    for _ in range(1_000_000):
        e.__init__(1)

threads = [threading.Thread(target=worker) for _ in range(2)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Gives on a 3.16 free-threaded debug build:

../cpython/Include/refcount.h:617: _Py_NegativeRefcount: Assertion failed: object has negative ref count
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x2000204c3b0
object refcount : 0
object type     : 0x5623737ad5c8
object type name: tuple
object repr     : <refcnt 0 at 0x2000204c3b0>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Stack (most recent call first):
  File "/<home>/projects/python/setref_bug.py", line 8 in worker
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1161 in run
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1219 in _bootstrap_inner
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1181 in _bootstrap
[1]    397225 abort (core dumped)  ./build/python setref_bug.py

This arrives because Py_SETREF copies the ref and stores a new new one as two separate non-atomic steps. So two threads can do:

T1: old = *slot
T2: old = *slot      // same object as T1
T1: *slot = a
T2: *slot = b        // a's reference is lost, so a leaks
T1: Py_XDECREF(old)
T2: Py_XDECREF(old)  // the slot's one reference to old is dropped twice

I ran into this trying to build a cache which swapped an old value for a new value with Py_SETREF and finding it sometimes resulted in a double free when two threads swapped the value simultaneously (And ended up thinking they had the last reference).

CPython versions tested on:

3.16, CPython main branch

Operating systems tested on:

Linux

Vorherrschende Sprache
Python
Sterne
78.2k
Forks
36.6k
Ø Merge
19 Std. 57 Min.
Gemergte PRs (30 T.)
630

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus python/cpython

Alle Issues in python/cpython

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.