Avoid stop-the-world when `__abstractmethods__` does not change the abstract flag

Aberta
  • Algo errado?
#161,744 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
48/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
c, python
Domínio
backend, performance

Direção de pesquisa

Start by locating type_set_abstractmethods() in CPython’s C implementation and inspect how it reads and changes Py_TPFLAGS_IS_ABSTRACT around the stop-the-world section. The issue provides the proposed condition and a benchmark covering concrete, protocol, abstract, and plain classes; use those cases to assess the change. Done means unchanged abstract state avoids stopping the world while state changes preserve correct behavior.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

interpreter-core performance topic-free-threading type-feature
Summary

In a free-threaded CPython build, type_set_abstractmethods() stops the worldevery time ABCMeta writes __abstractmethods__, even when the write does notchange Py_TPFLAGS_IS_ABSTRACT.

ABCMeta performs this write for every newly created class. For a concrete subclass of an ABC, the value is normally an empty set, so the abstract flag is already clear. Nevertheless, the current implementation briefly stops all other Python threads.

Benchmark
import argparse, gc, statistics, sys, threading, time, typing
from abc import ABC, abstractmethod
from collections.abc import Sized

def make(kind, n):
    if kind == "concrete":           
        for _ in range(n):
            class X(Sized):
                def __len__(self): return 0
    elif kind == "protocol":          
        for _ in range(n):
            class P(typing.Protocol):
                def m(self) -> int: ...
            class C(P):
                def m(self) -> int: return 0
    elif kind == "abstract":
        for _ in range(n):
            class Y(ABC):
                @abstractmethod
                def m(self): ...
    elif kind == "plain":
        for _ in range(n):
            class Z: pass
    else:
        raise SystemExit(f"unknown kind {kind}")

def one_round(kind, k, n):
    barrier = threading.Barrier(k + 1)
    def worker():
        barrier.wait()
        make(kind, n)
    ts = [threading.Thread(target=worker) for _ in range(k)]
    for t in ts: t.start()
    barrier.wait()
    t0 = time.perf_counter()
    for t in ts: t.join()
    return time.perf_counter() - t0

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--kind", default="concrete")
    ap.add_argument("--threads", type=int, default=1)
    ap.add_argument("--n", type=int, default=2000)
    ap.add_argument("--repeat", type=int, default=5)
    ap.add_argument("--no-gc", action="store_true")
    a = ap.parse_args()
    if a.no_gc:
        gc.disable()
    make(a.kind, 50)                  
    per = [one_round(a.kind, a.threads, a.n) / (a.n * a.threads) * 1e6
           for _ in range(a.repeat)]
    gil = getattr(sys, "_is_gil_enabled", lambda: True)()
    print(f"{a.kind},{a.threads},{a.n},{statistics.median(per):.2f},{min(per):.2f},{gil}")

if __name__ == "__main__":
    main()

Multiple threads run make_classes(2000) concurrently. The benchmark reports the median time per created class over three repetitions. It also includes three controls:

  • protocol: a typing.Protocol and a concrete subclass;
  • abstract: an ABC subclass that remains abstract;
  • plain: a class that does not use ABCMeta.
Implementation

In type_set_abstractmethods(), compare the requested abstract state with the current Py_TPFLAGS_IS_ABSTRACT flag before entering the stop-the-world section:

if (abstract != ((type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) != 0)) {
    type_lock_prevent_release(&pinned);
    types_stop_world();
    if (abstract)
        type_add_flags(type, Py_TPFLAGS_IS_ABSTRACT);
    else
        type_clear_flags(type, Py_TPFLAGS_IS_ABSTRACT);
    types_start_world();
    type_lock_allow_release(&pinned);
}
Performance results

Each result below is the median of five outer rounds. Each benchmark invocation
uses three inner repetitions and creates 2000 classes per thread.

Case Threads Baseline Patched Speedup Spread, baseline / patched
concrete 1 6.90 6.46 1.07x ±24% / ±14%
concrete 2 920.77 6.06 151.94x ±15% / ±11%
concrete 4 704.21 6.18 113.95x ±3% / ±8%
concrete 8 490.60 7.18 68.33x ±6% / ±11%
concrete 16 348.28 8.45 41.22x ±6% / ±6%
protocol 1 22.62 21.65 1.04x ±7% / ±5%
protocol 2 2801.11 1038.10 2.70x ±4% / ±2%
protocol 4 2463.42 1111.75 2.22x ±1% / ±0%
protocol 8 1963.49 1028.07 1.91x ±1% / ±1%
protocol 16 1397.80 783.15 1.78x ±3% / ±4%
abstract 1 6.62 6.93 0.96x ±16% / ±12%
abstract 2 885.09 894.81 0.99x ±10% / ±10%
abstract 4 636.98 632.91 1.01x ±1% / ±3%
abstract 8 445.71 461.46 0.97x ±6% / ±5%
abstract 16 329.88 333.72 0.99x ±9% / ±3%
plain 1 3.74 3.95 0.95x ±20% / ±39%
plain 2 3.46 3.67 0.94x ±13% / ±11%
plain 4 3.91 3.98 0.98x ±12% / ±7%
plain 8 4.16 4.32 0.96x ±7% / ±7%
plain 16 4.16 4.14 1.00x ±13% / ±3%
Linked PRs
  • gh-161745
Linguagem predominante
Python
Estrelas
78.2k
Forks
36.6k
Merge médio
19h 57min
PRs com merge (30d)
630

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de python/cpython

Todas as issues de python/cpython

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.