sys.monitoring callbacks (coverage, etc.) cause workflow sandbox hang on Python 3.14

Bắt đầu với entry point của sandbox importer có tên Importer.applied() được nêu trong issue và tái hiện MRE gồm hai tệp trên Python 3.14 với coverage. Theo dõi trạng thái monitoring trong khi workflow thực thi, sau đó xác minh rằng MRE hoàn tất thay vì bị treo và các bài kiểm thử SDK liên quan đều đạt. Issue cho biết một pull request được mong đợi, vì vậy hãy kiểm tra xem có công việc trùng lặp nào trước khi bắt đầu hay không.

hey guys I just want you to know I don't know what any of the words below mean (they were not in the bible)

the reason I'm opening this is because the pydantic-ai test suite was hanging in our PR to support python 3.14 https://github.com/pydantic/pydantic-ai/pull/4138

the hang came from the temporal tests, so me and opus started investigating and though it was very difficult we were finally I able to produce an MRE

so what you'll read (if you read it) below is opus' explanation, I'll put up a solution forward in a moment that I verified worked using the MRE and also running the test again with the patched temporal sdk

---

summary

on python 3.14, coverage switched from sys.settrace to sys.monitoring (PEP 669). sys.monitoring callbacks are global — they fire on all threads, including the workflow thread pool thread where the sandbox importer is active.

coverage's branch callback (sysmon_branch_either) lazily imports coverage.python the first time a BRANCH event fires. when this happens inside the sandbox, the import chain hits platform.python_implementation() which is restricted → RestrictedWorkflowAccessError → workflow task fails repeatedly → process hangs.

causal chain

greet() → `if name:` (BRANCH event)
→ coverage.sysmon.sysmon_branch_either
→ coverage.sysmon.get_multiline_map
→ coverage.parser.PythonParser.__init__
→ `from coverage.python import get_python_source`  ← lazy import
→ sandbox intercepts (coverage not in passthrough list)
→ coverage.__init__ → coverage.control → coverage.env
→ `platform.python_implementation()` ← restricted in sandbox
→ RestrictedWorkflowAccessError

this affects any sys.monitoring-based tool, not just coverage. any tool that registers monitoring callbacks and triggers a lazy import can hit this.

MRE

two files — the workflow must be in a separate module (not __main__) so the sandbox imports it normally and the branch callback fires during workflow execution rather than during import.

mre_workflow.py

from temporalio import workflow


def greet(name: str) -> str:
    if name:
        return f"Hello, {name}!"
    return "Hello!"


@workflow.defn
class SimpleWorkflow:
    @workflow.run
    async def run(self, name: str) -> str:
        return greet(name)

mre_coverage.py

import asyncio
import sys

from temporalio.testing import WorkflowEnvironment
from temporalio.worker import Worker

from mre_workflow import SimpleWorkflow


async def main() -> None:
    async with await WorkflowEnvironment.start_local() as env:
        task_queue = "mre-task-queue"
        async with Worker(
            env.client,
            task_queue=task_queue,
            workflows=[SimpleWorkflow],
        ):
            result = await asyncio.wait_for(
                env.client.execute_workflow(
                    SimpleWorkflow.run,
                    "world",
                    id="mre-workflow",
                    task_queue=task_queue,
                ),
                timeout=10,
            )
            print(f"Result: {result}")
            assert result == "Hello, world!", f"Unexpected: {result}"
            print("OK")


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except (asyncio.TimeoutError, TimeoutError):
        print(
            f"HANG DETECTED: workflow timed out (Python {sys.version_info.major}.{sys.version_info.minor})",
            file=sys.stderr,
        )
        sys.exit(1)

setup & reproduce

uv venv --python 3.14 .venv314
uv pip install --python .venv314/bin/python "temporalio[testing]" coverage

# hangs:
.venv314/bin/coverage run --source=. --branch mre_coverage.py

# works (no coverage):
.venv314/bin/python mre_coverage.py

# works (3.13 + coverage):
# uv venv --python 3.13 .venv313
# uv pip install --python .venv313/bin/python "temporalio[testing]" coverage
# .venv313/bin/coverage run --source=. --branch mre_coverage.py

why previous attempts at reproducing failed

• workflow with no branching code → branch callback never fires
• workflow defined in __main__ → sandbox re-imports it as __temporal_main__, and coverage's branch callback fires during import (before sandbox activation) rather than during workflow execution

environment

• python 3.14.2
• temporalio 1.20.0
• coverage 7.13.4

suggested fix

suspend sys.monitoring events inside Importer.applied(). save and clear all events on enter, restore on exit. this prevents any monitoring callback from firing inside the sandbox regardless of which tool registered it.

this is better than adding coverage to passthrough because:

• it's general — fixes all sys.monitoring-based tools
• monitoring callbacks are non-deterministic external state that shouldn't fire inside the sandbox anyway
• no user configuration needed

I'll put up a PR with this approach.