Synchronous Firestore call unbinds the caller thread's `contextvars` Context (breaks Flask request context) under gunicorn sync worker

オープン
#17,821 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
45/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
静か
技術スタック
flask, grpc, python
領域
backend, databases

調査の方向性

まず、Flask 2.3 と gunicorn sync workers の下で同期 get()/batch.commit() フローを再現し、その後 grpc/_channel.py の _run_channel_spin_thread、ForkManagedThread、segregated_call の周辺を調査します。動作を grpc/grpc#40123 および google/cloud/firestore_v1 の Firestore クライアント呼び出し箇所と比較します。コンテキストの変更が gRPC 全体に及ぶものか Firestore 固有のものかを特定し、サポートされる修正または回避策を文書化できれば完了です。

索引モデルが issue の本文から書いたものです。

説明

api: firestore priority: p3 type: question
Determine this is the right repository
  • I determined this is the correct repository in which to report this bug.
Summary of the issue

A synchronous Firestore call (gRPC transport) made from inside a request-handling
thread switches/unbinds that thread's contextvars Context partway through. Any
framework that stores per-request state in a ContextVar (Flask 2.3+ keeps its request
context in _cv_request) sees that state disappear on the same thread immediately
after the Firestore call. In our Flask app this surfaces as:

RuntimeError: Working outside of request context

raised during response finalization — a 500 to the client, even though the view code
completed successfully.

Firebase support directed us here, and pointed to a similar regression in the gRPC repo:
grpc/grpc#40123 ("contextvar loss when using thread pool executors in newer gRPC
versions"). Filing so the Cloud Firestore team can determine whether this is global gRPC
behavior or specific to the Firestore client's transport.

Environment

  • Python: 3.9
  • google-cloud-firestore: 2.18.0
  • firebase-admin: 6.5.0 (thin wrapper over the client above)
  • google-api-core: 2.19.2
  • grpcio / grpcio-status: 1.66.1
  • Transport: default (gRPC)
  • Web stack: Flask 2.3.0 / Werkzeug 3.0.3, served by gunicorn sync worker
    (one request at a time per worker process; stable OS thread)
  • Client init: module-level firestore.client() (channel created lazily on first RPC)

Observed behavior

We instrumented contextvars.copy_context and logged has_request_context() (hrc)
with a full stack at each call. Within a single request, on the same OS thread:

(A) First Firestore call — hrc is True (context healthy). Fires at channel creation:

grpc/_channel.py:1761  _run_channel_spin_thread -> cygrpc.ForkManagedThread(target=channel_spin)
grpc/_channel.py:1821  create -> _run_channel_spin_thread(state)
grpc/_channel.py:1414  __call__ -> self._managed_call(...)
google/api_core/grpc_helpers.py:164  error_remapped_callable
google/api_core/timeout.py / retry_unary.py / gapic_v1/method.py
google/cloud/firestore_v1/services/firestore/client.py:1154  batch_get_documents
google/cloud/firestore_v1/document.py:402  get()          # <- our doc_ref.get()
... application view ... gunicorn sync worker.handle_request
[hrc=True]

(B) A later Firestore call in the SAME request — hrc is now False (context gone):

grpc/_channel.py:1146  _blocking -> self._channel.segregated_call(...)
grpc/_channel.py:1178  __call__ -> self._blocking(...)
google/api_core/grpc_helpers.py:76  error_remapped_callable
google/api_core/timeout.py / retry_unary.py / gapic_v1/method.py
google/cloud/firestore_v1/services/firestore/client.py:1372  commit
google/cloud/firestore_v1/batch.py:59  commit()           # <- our batch.commit()
... same application/gunicorn stack ...
[hrc=False]

Between (A) and (B) our code runs only in-process work (SQLAlchemy queries + a list
comprehension) — nothing that touches contextvars, spawns threads, or uses
asyncio/greenlets. The only machinery that manipulates context between the two points
is the Firestore/gRPC transport (_run_channel_spin_thread / cygrpc.ForkManagedThread
at channel creation, and segregated_call on subsequent RPCs).

Expected vs actual

  • Expected: a synchronous Firestore RPC leaves the calling thread's contextvars
    Context unchanged.
  • Actual: after the RPC, the calling thread is running in a different Context; any
    ContextVar set before the call (e.g. Flask's _cv_request) reads as unset.

Reproduction

Flask 2.3 app served by gunicorn sync worker. In a view: log has_request_context(),
do a synchronous Firestore read+write (doc_ref.get() then batch.commit()), log
has_request_context() again. On the first Firestore-touching request after a worker
starts (fresh gRPC channel), the second check is False and Flask raises "Working outside
of request context" during response finalization. Minimal standalone repro available on
request.

Questions

  1. Is the Firestore Python client (or the underlying gRPC transport) expected to alter
    the calling thread's contextvars Context during a synchronous RPC?
  2. Is this the same root cause as grpc/grpc#40123, or specific to the Firestore client's
    ForkManagedThread / segregated_call usage?
  3. Supported ways to avoid it:
    • Pre-creating/warming the gRPC channel at process (post-fork) startup — does that
      help, or does segregated_call disturb context on every RPC regardless of channel
      age?
    • Using the REST transport (transport="rest") — is that a supported path to avoid
      the gRPC thread/context machinery entirely?
    • A known-good grpcio version to pin below the regression, if this is version-specific?

Current workaround

We run all Firestore calls on a short-lived daemon thread so the context churn lands on
a disposable thread and the request thread's context stays intact. This resolves the
500s, but we would prefer a supported fix rather than thread-isolating every call.

Impact

Intermittent 500s (correlated with the first Firestore call per worker / gRPC channel
creation) on any endpoint that makes a synchronous Firestore call, in production.

主要言語
Python
スター
5.4k
フォーク
1.8k
平均マージ
2日 7時間
マージ済み PR(30日)
173

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

googleapis/google-cloud-python のほかの issue

googleapis/google-cloud-python の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。