[Feature]: Honor OTEL_TRACES_SAMPLER for durable spans

Abierto
  • ¿Algo no está bien?
#887 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
observability

Línea de trabajo

Start with the current decision path in durable_sampling.py and review the related documentation issue #295 and the shared sampling contract. Because this issue requires an RFC, first settle the cross-language policy, parent-context inputs, defaults, and migration approach before implementation. Done means both Python views meet the policy matrix and regression criteria, with coordinated documentation and language behavior.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement pkg:otel project
What would you like?

Make both Python durable OpenTelemetry views honor the configured provider sampler selected through the standard OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG settings in supported ADOT/autoconfiguration setups.

Currently, an explicit propagated X-Ray Sampled=0/1 decision bypasses the configured sampler and is enforced for durable spans. Consequently, OTEL_TRACES_SAMPLER=always_off can drop ADOT handler/client spans while the durable plugin still exports Workflow, Invocation, operation, and attempt spans when the upstream header contains Sampled=1. This is the documented current behavior; this feature requests changing that contract so the selected sampler determines whether to inherit the parent decision.

Support the standard policies with their OpenTelemetry semantics:

Configured sampler Parent sampled (Sampled=1) Parent not sampled (Sampled=0) No parent supplied to sampler
always_on Sample Sample Sample
always_off Drop Drop Drop
traceidratio Decide from trace ID and ratio Decide from trace ID and ratio Decide from trace ID and ratio
parentbased_always_on Sample Drop Sample
parentbased_always_off Sample Drop Drop
parentbased_traceidratio Sample Drop Decide from trace ID and ratio

Use OTEL_TRACES_SAMPLER_ARG for the ratio through the provider's existing configuration mechanism. A valid parent with an unset sampled bit is different from having no parent; retain OpenTelemetry ParentBased semantics for that case.

Possible Implementation

Use the provider's resolved sampler rather than adding a separate environment parser in the durable plugin. Preserve programmatically configured providers and their ownership/lifecycle; document which supported setup resolves environment configuration and how explicit provider configuration takes precedence.

Define the stable parent context supplied to the sampler. ParentBased policies must see the applicable propagated parent, while a root policy must receive no parent. Do not always evaluate ParentBased at root, or feed a synthesized sampled ancestor back into a root decision. Resolve the durable sampling decision consistently and apply it to every SDK-created span; ordinary application and ADOT spans retain their provider behavior.

Sampling must remain deterministic for a durable execution: use its stable canonical trace ID with policy/configuration unchanged across retries and reinvocations. Random, time-dependent, load-dependent, or mutable-state-dependent custom decisions remain unsupported. This feature does not request sampling-decision persistence or make in-memory caching an additional supported sampling mode.

Acceptance criteria:

  • Both Execution and Invocation views implement the policy matrix through supported provider/autoconfiguration paths, including OTEL_TRACES_SAMPLER_ARG.
  • Define and document behavior when the environment variables are unset, invalid, or superseded by programmatic provider configuration; reuse the provider's validation/default behavior where applicable.
  • Test explicit upstream sampled/not-sampled contexts, a valid parent with no sampled flag, and no usable parent. Assert direct-sampler versus ParentBased behavior separately.
  • With always_off and upstream Sampled=1, no durable Workflow, Invocation, operation, attempt, or synthetic-root spans are exported in either the initial or resumed invocation.
  • Deterministic ratio tests exercise trace IDs on both sides of the threshold and initial invocation plus resume/retry. Every durable span follows the execution's resolved decision; completed user work is skipped on replay. Do not require exactly half of a small random batch to be sampled.
  • Preserve provider ownership, ordinary application/ADOT span sampling, trace identity, and context cleanup. No persistent sampling-decision store is introduced.
  • Keep a regression for sampled durable spans encountering a known dropped same-trace ambient handler parent. Honoring always_off alone does not establish that every missing-parent case is fixed. Define the appropriate fallback and distinguish exporter evidence from X-Ray backend/console qualification.
  • Update the Python README and shared sampling contract with compatibility/migration guidance, and coordinate JavaScript/Java behavior and applicable language-neutral conformance requirements before implementing inconsistent per-language precedence.
Is this a breaking change?

Yes. This changes documented trace sampling behavior: explicit upstream Sampled=0/1 currently takes precedence regardless of the configured sampler. Describe migration to parentbased_* for applications that want to retain upstream sampling decisions. Define the unset/default behavior explicitly.

Does this require an RFC?

Yes. Settle the cross-language sampling contract, parent-context inputs, defaults, and compatibility approach before implementation.

Additional Context
  • Related documentation issue: aws/aws-durable-execution-docs#295. That issue documents the existing support boundary and unconditional upstream precedence; this issue requests a runtime feature and requires coordinated updates to that contract.
  • Current Python decision path: durable_sampling.py.
  • Local reproduction on 44b9f4d1 and 4625d01b used an AlwaysOff provider, explicit upstream sampled context, and a same-trace handler span dropped by that provider. Across both views and initial/resumed invocations, all durable spans—including both Invocation spans—were exported sampled. Invocation parent IDs referred to the unexported handler spans. Removing that ambient span made Invocation fall back to the propagated remote parent.
  • Those checks establish local sampler and parent-selection behavior. They did not exercise the ADOT layer binary, the X-Ray exporter/backend, or console rendering. Retain real deployment/export evidence when qualifying the supported ADOT path.

The parent-selection case is a related regression requirement, not evidence that the durable sampler itself dropped Invocation spans. The separate replay-completion issue #854 concerns unvisited terminal operations and does not track this configuration behavior.

Lenguaje dominante
Python
Estrellas
58
Forks
32
Merge medio
1 d 6 h
PR fusionados (30 d)
80

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de aws/aws-durable-execution-sdk-python

Todos los issues de aws/aws-durable-execution-sdk-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.