Firestore's @transactional is not thread safe

オープン
#19,060 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

@itsrayeedscodespace がすでに取り組んでいます。

2026年11月13日 から。

  • #19073 @itsrayeedscodespace による — オープン

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
65/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
databases

調査の方向性

The reproduction is in main.py and uses firestore.transactional with two concurrent Transaction instances. Start by tracing the decorator's retry handling and where retry transaction IDs are stored; then check the related Firestore transaction tests. Done when concurrent decorated calls keep their retry IDs isolated and a regression test reproduces the race.

索引モデルが issue の本文から書いたものです。

説明

triage me type: bug
Determine this is the right repository
  • I determined this is the correct repository in which to report this bug.
Summary of the issue

Context
I use @firestore.transactional to run transactions in python

Expected Behavior:
I'd expect the wrapper to be thread safe and allow multiple concurrent calls since it's a method decorator.

Actual Behavior:
On retries, transactions end up with each other's retry ids, which causes failures.

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 276, in __call__
    response, ignored_call = self._with_call(
                             ^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 331, in _with_call
    return call.result(), call
           ^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 447, in result
    raise self
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 314, in continuation
    response, call = self._thunk(new_method).with_call(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/grpcext/_interceptor.py", line 77, in with_call
    return self._interceptor.intercept_unary(request, metadata, client_info, invoker)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 177, in intercept_unary
    return self._intercept(request, metadata, client_info, invoker)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 158, in _intercept
    raise exc
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 144, in _intercept
    result = invoker(request, metadata)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/grpcext/_interceptor.py", line 67, in invoker
    return self._base_callable.with_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 1182, in with_call
    return _end_unary_response_blocking(state, call, True, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 999, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "The referenced transaction has expired or is no longer valid."
	debug_error_string = "INVALID_ARGUMENT:The referenced transaction has expired or is no longer valid."
API client name and version

google-cloud-firestore 2.34.1

Reproduction steps: code

file: main.py

"""Repro: concurrent calls of one @firestore.transactional function share its retry id.

Call A's commit is aborted while call B is in flight. A retries with
BeginTransaction(retry_transaction=<B's id>), which in production invalidates B, whose commit then
fails with "The referenced transaction has expired or is no longer valid".

    pip install google-cloud-firestore
    python transactional_race_repro.py
"""

import threading
from concurrent.futures import ThreadPoolExecutor

from google.api_core import exceptions
from google.auth.credentials import AnonymousCredentials
from google.cloud import firestore

WAIT_SECONDS = 5


class FakeRpcTransaction(firestore.Transaction):
    def __init__(self, name, aborted_commits):
        super().__init__(
            firestore.Client(project="repro", credentials=AnonymousCredentials())
        )
        self.name = name
        self.aborted_commits = aborted_commits
        self.begun_with_retry_ids = []

    def _begin(self, retry_id=None):
        self.begun_with_retry_ids.append(retry_id)
        self._id = f"{self.name}-{len(self.begun_with_retry_ids)}".encode()

    def _commit(self):
        if self.aborted_commits > 0:
            self.aborted_commits -= 1
            raise exceptions.Aborted("Contention")
        self._clean_up()
        return []

    def _rollback(self):
        self._clean_up()


@firestore.transactional
def update_in_transaction(transaction, began, wait_for):
    # Release the GIL, as a network call in a transaction would
    began.set()
    assert wait_for.wait(WAIT_SECONDS)


def main():
    a = FakeRpcTransaction("A", aborted_commits=1)
    b = FakeRpcTransaction("B", aborted_commits=0)
    a_began, b_began, a_finished = (threading.Event() for _ in range(3))
    with ThreadPoolExecutor(max_workers=2) as executor:
        # 1. A begins and is in flight.
        a_call = executor.submit(update_in_transaction, a, a_began, b_began)
        assert a_began.wait(WAIT_SECONDS)
        # 2. B begins while A is in flight, resetting the shared retry id and then setting it to B's.
        b_call = executor.submit(update_in_transaction, b, b_began, a_finished)
        # 3. A's commit aborts, so A retries with the shared retry id, which is now B's transaction id.
        a_call.result(timeout=WAIT_SECONDS)
        a_finished.set()
        b_call.result(timeout=WAIT_SECONDS)

    print(f"google-cloud-firestore {firestore.__version__}")
    print(f"A began with retry ids {a.begun_with_retry_ids}")
    assert a.begun_with_retry_ids == [None, b"A-1"], (
        f"A's retry began as a retry of B's transaction {a.begun_with_retry_ids[1]!r}, "
        "not of its own b'A-1'"
    )


if __name__ == "__main__":
    main()
Reproduction steps: supporting files

file: mydata.csv

alpha,1,3
beta,2,5
Reproduction steps: actual results

file: output.txtmydata.csv

Calculated: foo
Reproduction steps: expected results

file: output.txtmydata.csv

Calculated: bar
OS & version + platform

OSX

Python environment

3.12.14

Python dependencies

No response

Additional context

No response

主要言語
Python
スター
5.4k
フォーク
1.8k
平均マージ
2日 20時間
マージ済み PR(30日)
140

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

googleapis/google-cloud-python のほかの issue

googleapis/google-cloud-python の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。