[Bug]: OTel ignores the invocation-local X-Ray header on Lambda Managed Instances
メンテナーはふだん 1 日以内に返信
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 38/100
- issue の種類
- バグ
- 明瞭さ
- 明確に書かれている
- 活発さ
- 活発
- 技術スタック
- aws, java
調査の方向性
Start with otel-plugin/src/main/java/software/amazon/lambda/durable/otel/XRayContextExtractor.java, sdk/src/main/java/software/amazon/lambda/durable/plugin/InvocationInfo.java, and the invocation/plugin entry points; run the supplied LmiTraceCarrierProbe first. Trace how invocation data reaches worker threads, then verify preserved Root, Parent, and sampling values for overlapping, replay, and resume cases while retaining ordinary-Lambda behavior and adding deployed LMI validation.
索引モデルが issue の本文から書いたものです。
説明
Expected Behavior
The Java OTel integration should preserve the trace ID, parent and sampling decision provided by the LMI invocation context. This information must remain scoped to the invocation, including when the SDK dispatches user code to another thread.
Actual Behavior
Java LMI supplies Context.getXrayTraceId() and puts the header in the runtime ThreadLocal carrier. It does not update the ordinary Lambda global system property. The default XRayContextExtractor reads only com.amazonaws.xray.traceHeader, then _X_AMZN_TRACE_ID; neither path reaches the LMI header.
With a valid Sampled=0 header placed in both an invocation Context and the runtime-style SdkInternalThreadLocal carrier, the published extractor returns null. Both OTel views then export sampled Invocation/Workflow on a different trace when the fallback sampler is AlwaysOn. The ordinary-Lambda system-property control correctly extracts NOT_SAMPLED and exports no spans.
Steps to Reproduce
Use software.amazon.lambda.durable:aws-durable-execution-sdk-java-plugin-otel:2.2.1 and io.opentelemetry:opentelemetry-sdk-testing:1.65.0 with their transitive dependencies in a scratch Maven project. Run LmiTraceCarrierProbe below with _X_AMZN_TRACE_ID unset. The probe uses the real published plugins/exporter and the same ThreadLocal key used by the runtime; it makes no AWS calls.
The Context/ThreadLocal fixture models LMI input. It does not start a real Lambda runtime. Both the missing-header observation and the ordinary-carrier control are asserted.
Reproduction code
import java.lang.reflect.Proxy;
import java.time.Instant;
import java.util.*;
import com.amazonaws.services.lambda.runtime.Context;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter;
import software.amazon.awssdk.utilslite.SdkInternalThreadLocal;
import software.amazon.lambda.durable.otel.*;
import software.amazon.lambda.durable.plugin.*;
public class LmiTraceCarrierProbe {
static final String HEADER = "Root=1-6955b900-123456789012345678901234;Parent=1234567890123456;Sampled=0";
static final String EXPECTED_TRACE = "6955b900123456789012345678901234";
static final String PROPERTY = "com.amazonaws.xray.traceHeader";
static final String KEY = "AWS_LAMBDA_X_TRACE_ID";
public static void main(String[] args) throws Exception {
if (System.getenv("_X_AMZN_TRACE_ID") != null) throw new AssertionError("Probe requires no environment header");
Context runtimeContext = (Context) Proxy.newProxyInstance(Context.class.getClassLoader(), new Class<?>[]{Context.class},
(proxy, method, values) -> {
if (method.getName().equals("getXrayTraceId")) return HEADER;
throw new UnsupportedOperationException(method.getName());
});
var results = new ArrayList<Map<String,Object>>();
for (boolean ordinary : new boolean[]{false,true}) {
System.clearProperty(PROPERTY);
SdkInternalThreadLocal.remove(KEY);
if (ordinary) System.setProperty(PROPERTY, HEADER);
else SdkInternalThreadLocal.put(KEY, HEADER);
var extracted = new XRayContextExtractor().extract();
if (ordinary && (extracted == null || !EXPECTED_TRACE.equals(extracted.traceId()) || extracted.sampling() != ExtractedContext.Sampling.NOT_SAMPLED))
throw new AssertionError("Ordinary carrier failed");
if (!ordinary && (extracted != null || !HEADER.equals(runtimeContext.getXrayTraceId()) || !HEADER.equals(SdkInternalThreadLocal.get(KEY))))
throw new AssertionError("LMI fixture or expected extractor result differs");
for (boolean executionView : new boolean[]{true,false}) {
var exporter = InMemorySpanExporter.create();
var builder = SdkTracerProvider.builder().setSampler(Sampler.alwaysOn())
.addSpanProcessor(SimpleSpanProcessor.create(exporter));
var config = OtelPluginConfig.builder().enableMdc(false).build();
DurableExecutionPlugin plugin = executionView ? new ExecutionOtelPlugin(builder,config) : new InvocationOtelPlugin(builder,config);
var arn="arn:aws:lambda:us-west-2:123456789012:function:probe:1/durable-execution/probe/id";
plugin.onInvocationStart(new InvocationInfo("request",arn,true,Instant.parse("2026-10-02T00:00:00Z")));
plugin.onInvocationEnd(new InvocationEndInfo("request",arn,true,InvocationStatus.SUCCEEDED,null));
var spans = exporter.getFinishedSpanItems().stream().map(s -> Map.of(
"name",s.getName(),"traceId",s.getTraceId(),"sampled",s.getSpanContext().isSampled())).toList();
if (ordinary && !spans.isEmpty()) throw new AssertionError("Upstream Sampled=0 must suppress exports");
if (!ordinary && (spans.size()!=2 || exporter.getFinishedSpanItems().stream().anyMatch(s -> EXPECTED_TRACE.equals(s.getTraceId()) || !s.getSpanContext().isSampled())))
throw new AssertionError("Unexpected LMI-path observation");
var record = new LinkedHashMap<String,Object>();
record.put("carrier",ordinary?"ordinary Lambda system property":"LMI context plus runtime ThreadLocal");
record.put("plugin",plugin.getClass().getSimpleName());
record.put("inputSampling","NOT_SAMPLED");
record.put("extractor",extracted==null?null:Map.of("traceId",extracted.traceId(),"sampling",extracted.sampling().name()));
record.put("spans",spans);
results.add(record);
}
}
System.clearProperty(PROPERTY);
SdkInternalThreadLocal.remove(KEY);
System.out.println(new ObjectMapper().writeValueAsString(Map.of("sdkAndPluginVersion","2.2.1","results",results,"checksPassed",true)));
}
}
SDK Version
Core SDK / OTel plugin 2.2.1. Main rechecked at 78574e3051aac9b5ca00a0c1fd2b91eab72a9e15.
Java Version
Corretto 25.0.4.1
Is this a regression?
Unknown; no last-known-good version has been established.
Additional Context
Sources: Java LMI request-context contract, runtime carrier selection, default extractor, and plugin invocation information.
PR #721, checked at c7be25f4a441a9629a8337b22f577d3abf25640c, leaves the extractor unchanged. This is separate from shared plugin-state isolation. Python is not affected by this carrier change: its official LMI runtime continues to support the environment header.
Acceptance criteria
- Capture the header from the Lambda Context and pass it through invocation-local SDK/plugin data, accounting for worker-thread dispatch.
- Preserve Root, Parent and explicit sampling; retain ordinary Lambda compatibility and deterministic fallback for genuinely absent/invalid headers.
- Avoid global system-property/environment writes and test overlapping invocations with distinct headers plus replay/resume.
- Add deployed LMI validation; the observations here are local reproductions and source inspection.
Related cross-SDK tracking: https://github.com/aws/aws-durable-execution-sdk-js/issues/951
- 主要言語
- Java
- スター
- 28
- フォーク
- 13
- 平均マージ
- 2日 3時間
- マージ済み PR(30日)
- 44
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
aws/aws-durable-execution-sdk-java のほかの issue
-
documentation pkg:sdk
難易度 1/5 1〜3時間 初心者へのやさしさ 88/100
aws/aws-durable-execution-sdk-java#645 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
aws/aws-durable-execution-sdk-java#300 ·
メンテナーはふだん 1 日以内に返信
-
enhancement needs-triage
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
aws/aws-durable-execution-sdk-java#779 ·
メンテナーはふだん 1 日以内に返信
-
[Bug]: root handler instrumentation misses the canonical OTel execution context対応中かも このイシューにリンクされたプルリクエストがオープン中、またはマージ済みです。 オープンneeds-triage
難易度 5/5 1週間以上 初心者へのやさしさ 40/100
aws/aws-durable-execution-sdk-java#770 ·
メンテナーはふだん 1 日以内に返信
-
[Feature]: Propagate per-operation trace context for chained invokes対応中かも このイシューにリンクされたプルリクエストがオープン中、またはマージ済みです。 オープンenhancement needs-triage
難易度 5/5 1週間以上 初心者へのやさしさ 38/100
aws/aws-durable-execution-sdk-java#764 ·
メンテナーはふだん 1 日以内に返信
aws/aws-durable-execution-sdk-java の issue をすべて見る
似ている issue
-
難易度 1/5 1時間未満 初心者へのやさしさ 85/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
commonmark/commonmark-java#460 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
GoogleCloudPlatform/spring-cloud-gcp#4664 ·
メンテナーはふだん 1 日以内に返信
-
enhancement user story
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 86/100