Free-threading race betwen two Py_SETREF

Aperta
  • Qualcosa non va?
#161,714 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
22/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
c, python
Ambito
backend

Direzione di ricerca

The race is in the Py_SETREF and Py_XSETREF macros, which copy the old pointer, store the new one, and then decref in separate non-atomic steps. Run the reproducer with a free-threaded debug build and read the assertion at Include/refcount.h:617 to confirm the negative refcount on a tuple. Done would mean the reproducer runs clean under free-threading, which needs a design decision about atomic swap semantics first.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

interpreter-core topic-free-threading type-bug
Bug report

This can be triggered with existing Python objects that use Py_SETREF or Py_XSETREF such as Exceptions (although slightly contrived):

import threading

e = Exception()

def worker():
    for _ in range(1_000_000):
        e.__init__(1)

threads = [threading.Thread(target=worker) for _ in range(2)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Gives on a 3.16 free-threaded debug build:

../cpython/Include/refcount.h:617: _Py_NegativeRefcount: Assertion failed: object has negative ref count
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x2000204c3b0
object refcount : 0
object type     : 0x5623737ad5c8
object type name: tuple
object repr     : <refcnt 0 at 0x2000204c3b0>

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Stack (most recent call first):
  File "/<home>/projects/python/setref_bug.py", line 8 in worker
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1161 in run
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1219 in _bootstrap_inner
  File "/<home>/projects/python/cpython/Lib/threading.py", line 1181 in _bootstrap
[1]    397225 abort (core dumped)  ./build/python setref_bug.py

This arrives because Py_SETREF copies the ref and stores a new new one as two separate non-atomic steps. So two threads can do:

T1: old = *slot
T2: old = *slot      // same object as T1
T1: *slot = a
T2: *slot = b        // a's reference is lost, so a leaks
T1: Py_XDECREF(old)
T2: Py_XDECREF(old)  // the slot's one reference to old is dropped twice

I ran into this trying to build a cache which swapped an old value for a new value with Py_SETREF and finding it sometimes resulted in a double free when two threads swapped the value simultaneously (And ended up thinking they had the last reference).

CPython versions tested on:

3.16, CPython main branch

Operating systems tested on:

Linux

Lingua principale
Python
Stelle
78.2k
Fork
36.6k
Merge medio
19h 57m
PR unite (30g)
630

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di python/cpython

Tutte le issue di python/cpython

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.