[Feature]: Honor OTEL_TRACES_SAMPLER for durable spans

Aperta
  • Qualcosa non va?
#887 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
observability

Direzione di ricerca

Start with the current decision path in durable_sampling.py and review the related documentation issue #295 and the shared sampling contract. Because this issue requires an RFC, first settle the cross-language policy, parent-context inputs, defaults, and migration approach before implementation. Done means both Python views meet the policy matrix and regression criteria, with coordinated documentation and language behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement pkg:otel project
What would you like?

Make both Python durable OpenTelemetry views honor the configured provider sampler selected through the standard OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG settings in supported ADOT/autoconfiguration setups.

Currently, an explicit propagated X-Ray Sampled=0/1 decision bypasses the configured sampler and is enforced for durable spans. Consequently, OTEL_TRACES_SAMPLER=always_off can drop ADOT handler/client spans while the durable plugin still exports Workflow, Invocation, operation, and attempt spans when the upstream header contains Sampled=1. This is the documented current behavior; this feature requests changing that contract so the selected sampler determines whether to inherit the parent decision.

Support the standard policies with their OpenTelemetry semantics:

Configured sampler Parent sampled (Sampled=1) Parent not sampled (Sampled=0) No parent supplied to sampler
always_on Sample Sample Sample
always_off Drop Drop Drop
traceidratio Decide from trace ID and ratio Decide from trace ID and ratio Decide from trace ID and ratio
parentbased_always_on Sample Drop Sample
parentbased_always_off Sample Drop Drop
parentbased_traceidratio Sample Drop Decide from trace ID and ratio

Use OTEL_TRACES_SAMPLER_ARG for the ratio through the provider's existing configuration mechanism. A valid parent with an unset sampled bit is different from having no parent; retain OpenTelemetry ParentBased semantics for that case.

Possible Implementation

Use the provider's resolved sampler rather than adding a separate environment parser in the durable plugin. Preserve programmatically configured providers and their ownership/lifecycle; document which supported setup resolves environment configuration and how explicit provider configuration takes precedence.

Define the stable parent context supplied to the sampler. ParentBased policies must see the applicable propagated parent, while a root policy must receive no parent. Do not always evaluate ParentBased at root, or feed a synthesized sampled ancestor back into a root decision. Resolve the durable sampling decision consistently and apply it to every SDK-created span; ordinary application and ADOT spans retain their provider behavior.

Sampling must remain deterministic for a durable execution: use its stable canonical trace ID with policy/configuration unchanged across retries and reinvocations. Random, time-dependent, load-dependent, or mutable-state-dependent custom decisions remain unsupported. This feature does not request sampling-decision persistence or make in-memory caching an additional supported sampling mode.

Acceptance criteria:

  • Both Execution and Invocation views implement the policy matrix through supported provider/autoconfiguration paths, including OTEL_TRACES_SAMPLER_ARG.
  • Define and document behavior when the environment variables are unset, invalid, or superseded by programmatic provider configuration; reuse the provider's validation/default behavior where applicable.
  • Test explicit upstream sampled/not-sampled contexts, a valid parent with no sampled flag, and no usable parent. Assert direct-sampler versus ParentBased behavior separately.
  • With always_off and upstream Sampled=1, no durable Workflow, Invocation, operation, attempt, or synthetic-root spans are exported in either the initial or resumed invocation.
  • Deterministic ratio tests exercise trace IDs on both sides of the threshold and initial invocation plus resume/retry. Every durable span follows the execution's resolved decision; completed user work is skipped on replay. Do not require exactly half of a small random batch to be sampled.
  • Preserve provider ownership, ordinary application/ADOT span sampling, trace identity, and context cleanup. No persistent sampling-decision store is introduced.
  • Keep a regression for sampled durable spans encountering a known dropped same-trace ambient handler parent. Honoring always_off alone does not establish that every missing-parent case is fixed. Define the appropriate fallback and distinguish exporter evidence from X-Ray backend/console qualification.
  • Update the Python README and shared sampling contract with compatibility/migration guidance, and coordinate JavaScript/Java behavior and applicable language-neutral conformance requirements before implementing inconsistent per-language precedence.
Is this a breaking change?

Yes. This changes documented trace sampling behavior: explicit upstream Sampled=0/1 currently takes precedence regardless of the configured sampler. Describe migration to parentbased_* for applications that want to retain upstream sampling decisions. Define the unset/default behavior explicitly.

Does this require an RFC?

Yes. Settle the cross-language sampling contract, parent-context inputs, defaults, and compatibility approach before implementation.

Additional Context
  • Related documentation issue: aws/aws-durable-execution-docs#295. That issue documents the existing support boundary and unconditional upstream precedence; this issue requests a runtime feature and requires coordinated updates to that contract.
  • Current Python decision path: durable_sampling.py.
  • Local reproduction on 44b9f4d1 and 4625d01b used an AlwaysOff provider, explicit upstream sampled context, and a same-trace handler span dropped by that provider. Across both views and initial/resumed invocations, all durable spans—including both Invocation spans—were exported sampled. Invocation parent IDs referred to the unexported handler spans. Removing that ambient span made Invocation fall back to the propagated remote parent.
  • Those checks establish local sampler and parent-selection behavior. They did not exercise the ADOT layer binary, the X-Ray exporter/backend, or console rendering. Retain real deployment/export evidence when qualifying the supported ADOT path.

The parent-selection case is a related regression requirement, not evidence that the durable sampler itself dropped Invocation spans. The separate replay-completion issue #854 concerns unvisited terminal operations and does not track this configuration behavior.

Lingua principale
Python
Stelle
58
Fork
32
Merge medio
1g 6h
PR unite (30g)
80

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di aws/aws-durable-execution-sdk-python

Tutte le issue di aws/aws-durable-execution-sdk-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.