[Bug] Unable to run workflows with OpenTelemetry and ddtrace
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- observability
Research direction
Start with the TracingInterceptor and the Temporal sandbox using the minimal workflow reproduction described in the issue. Run it with the OpenTelemetry SDK at version 1.29 or later and compare it with a version before 1.29, then inspect the linked stack trace and upstream OpenTelemetry change. Done means the traced workflow runs without the sandbox rejecting os.environ.get.
Written by the indexing model from the issue text.
Description
What are you really trying to do?
I'm attempting to enable tracing on my worker using the TracingInterceptor and the tracer provided by ddtrace.
Describe the bug
A change upstream with the OpenTelemetry SDK seems to have caused an issue with Temporal's sandbox environment, causing an error stating os.environ.get isn't allowed in a workflow.
The relevant change on the OpenTelemetry side is here - if I run with a version prior to 1.29 then I no longer get an error.
Minimal Reproduction
A workflow implemented on a client with the TracingInterceptor attached and a valid OpenTelemetry SDK configured, in my case I'm using ddtrace with the DD_TRACE_OTEL_ENABLED environment variable set to true. From what I can tell though the call that isn't allowed in the sandbox is coming directly from the OpenTelemetry SDK and not the ddtrace patch.
Environment/Versions
- OS and processor: macOS Apple Silicon
- Temporal Version: SDK 1.9.0
- Running in Docker arm64
Additional context
You can see the stacktrace and error that's displayed on the Temporal UI here: https://gist.github.com/connected-bkiiskila/d11592cb86271b5f343a4d387097d418
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 241
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 49
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from temporalio/sdk-python
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
temporalio/sdk-python#1517 · 10 comments ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
temporalio/sdk-python#496 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
temporalio/sdk-python#1894 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
temporalio/sdk-python#1890 ·
-
[Bug] Local activity resolutions regrouped on replay since 1.32.0, delivering the wrong payload Open
Difficulty 4/5 3-5 days Newbie friendliness 52/100
temporalio/sdk-python#1881 · 1 comment ·
All issues in temporalio/sdk-python
Similar issues
-
essnmx good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
[Feature] 奇物选择添加优先级 Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
syfoud/Simulated_Scepter#174 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Giskard-AI/giskard-oss#2840 · 1 comment ·
-
A claim comment carrying the issue number is silently declined while the workflow reports success Openarea: repo bug perceived difficulty: 2
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
yeti-platform/yeti#1380 ·