Firestore's @transactional is not thread safe
维护者通常 2 天内回复
评估
调研方向
The reproduction is in main.py and uses firestore.transactional with two concurrent Transaction instances. Start by tracing the decorator's retry handling and where retry transaction IDs are stored; then check the related Firestore transaction tests. Done when concurrent decorated calls keep their retry IDs isolated and a regression test reproduces the race.
由索引模型根据 Issue 内容生成。
描述
Determine this is the right repository
- I determined this is the correct repository in which to report this bug.
Summary of the issue
Context
I use @firestore.transactional to run transactions in python
Expected Behavior:
I'd expect the wrapper to be thread safe and allow multiple concurrent calls since it's a method decorator.
Actual Behavior:
On retries, transactions end up with each other's retry ids, which causes failures.
Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 276, in __call__
response, ignored_call = self._with_call(
^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 331, in _with_call
return call.result(), call
^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 447, in result
raise self
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_interceptor.py", line 314, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/grpcext/_interceptor.py", line 77, in with_call
return self._interceptor.intercept_unary(request, metadata, client_info, invoker)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 177, in intercept_unary
return self._intercept(request, metadata, client_info, invoker)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 158, in _intercept
raise exc
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/_client.py", line 144, in _intercept
result = invoker(request, metadata)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/opentelemetry/instrumentation/grpc/grpcext/_interceptor.py", line 67, in invoker
return self._base_callable.with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 1182, in with_call
return _end_unary_response_blocking(state, call, True, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/layers/google.python.pip/pip/lib/python3.12/site-packages/grpc/_channel.py", line 999, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "The referenced transaction has expired or is no longer valid."
debug_error_string = "INVALID_ARGUMENT:The referenced transaction has expired or is no longer valid."
API client name and version
google-cloud-firestore 2.34.1
Reproduction steps: code
file: main.py
"""Repro: concurrent calls of one @firestore.transactional function share its retry id.
Call A's commit is aborted while call B is in flight. A retries with
BeginTransaction(retry_transaction=<B's id>), which in production invalidates B, whose commit then
fails with "The referenced transaction has expired or is no longer valid".
pip install google-cloud-firestore
python transactional_race_repro.py
"""
import threading
from concurrent.futures import ThreadPoolExecutor
from google.api_core import exceptions
from google.auth.credentials import AnonymousCredentials
from google.cloud import firestore
WAIT_SECONDS = 5
class FakeRpcTransaction(firestore.Transaction):
def __init__(self, name, aborted_commits):
super().__init__(
firestore.Client(project="repro", credentials=AnonymousCredentials())
)
self.name = name
self.aborted_commits = aborted_commits
self.begun_with_retry_ids = []
def _begin(self, retry_id=None):
self.begun_with_retry_ids.append(retry_id)
self._id = f"{self.name}-{len(self.begun_with_retry_ids)}".encode()
def _commit(self):
if self.aborted_commits > 0:
self.aborted_commits -= 1
raise exceptions.Aborted("Contention")
self._clean_up()
return []
def _rollback(self):
self._clean_up()
@firestore.transactional
def update_in_transaction(transaction, began, wait_for):
# Release the GIL, as a network call in a transaction would
began.set()
assert wait_for.wait(WAIT_SECONDS)
def main():
a = FakeRpcTransaction("A", aborted_commits=1)
b = FakeRpcTransaction("B", aborted_commits=0)
a_began, b_began, a_finished = (threading.Event() for _ in range(3))
with ThreadPoolExecutor(max_workers=2) as executor:
# 1. A begins and is in flight.
a_call = executor.submit(update_in_transaction, a, a_began, b_began)
assert a_began.wait(WAIT_SECONDS)
# 2. B begins while A is in flight, resetting the shared retry id and then setting it to B's.
b_call = executor.submit(update_in_transaction, b, b_began, a_finished)
# 3. A's commit aborts, so A retries with the shared retry id, which is now B's transaction id.
a_call.result(timeout=WAIT_SECONDS)
a_finished.set()
b_call.result(timeout=WAIT_SECONDS)
print(f"google-cloud-firestore {firestore.__version__}")
print(f"A began with retry ids {a.begun_with_retry_ids}")
assert a.begun_with_retry_ids == [None, b"A-1"], (
f"A's retry began as a retry of B's transaction {a.begun_with_retry_ids[1]!r}, "
"not of its own b'A-1'"
)
if __name__ == "__main__":
main()
Reproduction steps: supporting files
file: mydata.csv
alpha,1,3
beta,2,5
Reproduction steps: actual results
file: output.txtmydata.csv
Calculated: foo
Reproduction steps: expected results
file: output.txtmydata.csv
Calculated: bar
OS & version + platform
OSX
Python environment
3.12.14
Python dependencies
No response
Additional context
No response
- 主要语言
- Python
- 星标
- 5.4k
- 派生
- 1.8k
- 平均合并
- 2 天 20 小时
- 30 天内合并 PR
- 140
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
googleapis/google-cloud-python 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
googleapis/google-cloud-python#19099 ·
维护者通常 2 天内回复
-
firestore: field path with a quoted name ending in a backslash is misparsed可能已有人在做 @sebxt 于 28 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
googleapis/google-cloud-python#18771 ·
维护者通常 2 天内回复
-
api: spanner type: feature request
难度 2/5 1-3 小时 新手友好度 68/100
googleapis/google-cloud-python#18584 ·
维护者通常 2 天内回复
-
auth: `impersonated_credentials` loses `subject` on copy, so `with_scopes()` and `with_quota_project()` turn off domain wide delegation可能已有人在做 @Om-singhaI 于 58 天前认领。 未关闭auth priority: p2 type: bug
难度 2/5 1-3 小时 新手友好度 88/100
googleapis/google-cloud-python#18428 ·
维护者通常 2 天内回复
-
A client called `run_in_transaction` be stuck if Spanner is down可能已有人在做 @founderinaction 于 10 天前认领。 未关闭priority: p2 type: bug
难度 2/5 1-3 小时 新手友好度 72/100
googleapis/google-cloud-python#18375 · 1 条评论 ·
维护者通常 2 天内回复
查看 googleapis/google-cloud-python 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 64/100
-
难度 2/5 1-3 小时 新手友好度 76/100
btclib-org/btclib-wallet#1045 ·
维护者通常 1 天内回复
-
area-templates triage:bot-seen triage:needs-human
难度 2/5 1-3 小时 新手友好度 75/100
microsoft/aspire#22356 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
[Train] An elastic ScalingConfig(num_workers=(0, 0)) is accepted and the run waits forever可能已有人在做 @wguo7 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
ray-project/ray#68217 ·
维护者通常 2 天内回复