Spring Actuator created extra resources in APM
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- backend, observability-sre
Research direction
Compare tracing behavior between versions 1.53.0 and 1.55.0 for Spring Actuator requests, using the OperationHandler.handle entry point and the supplied DD_TRACE_DEBUG trace as the starting evidence. Reproduce the Kubernetes health-check request and verify that excluded actuator resources no longer create extra APM resources.
Written by the indexing model from the issue text.
Description
Tracer Version(s)
1.54.0, 1.55.0
Java Version(s)
25.0.36
JVM Vendor
Eclipse Adoptium / Temurin
Bug Report
After updating from 1.40.1 to 1.55.0, I noticed a lot of requests for a Resource named OperationHandler.handle inside APM on a Service. I eventually linked it to Spring Actuator requests, mostly from a Kubernetes health check.
We have ".*/actuator.*" set to be excluded at the Datadog Agent level in the cluster, but there seemed to be a new span created for each actuator call.
I gradually reverted the library version and once I got to 1.53.0, the issue went away.
Here's a bit of an excerpt for a single Trace from the logs with DD_TRACE_DEBUG=true:
2025-11-07 14:43:01.907 [tomcat-handler-60] DEBUG datadog.trace.agent.core.propagation.HttpCodec - Extract incomplete context TagContext{baggage={}, }
2025-11-07 14:43:01.907 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `null`->`null` with priority 0->0 for traceId=3536004845519906260 spanId=8720177333644726388
2025-11-07 14:43:01.908 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpan - Started span: DDSpan [ t_id=3536004845519906260, s_id=8720177333644726388, p_id=0 ] trace=edi-gateway-api/servlet.request/servlet.request tags={_dd.injection.mode=k8s, _dd.profiling.enabled=0, _dd.trace_span_attribute_schema=0, env=dev, language=jvm, process_id=1, runtime-id=297c889b-aaf3-4e12-a504-b8eb641f364b, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=0, forceKeep=false, links=[]
2025-11-07 14:43:01.908 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `null`->`GET /edi/api/actuator/health` with priority 0->1 for traceId=3536004845519906260 spanId=8720177333644726388
2025-11-07 14:43:01.909 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `GET /edi/api/actuator/health`->`GET /edi/api/actuator/health` with priority 1->1 for traceId=3536004845519906260 spanId=8720177333644726388
2025-11-07 14:43:01.910 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `GET /edi/api/actuator/health`->`GET /edi/api/actuator/health` with priority 1->3 for traceId=3536004845519906260 spanId=8720177333644726388
2025-11-07 14:43:01.913 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `GET /edi/api/actuator/health`->`GET /edi/api/actuator/health` with priority 3->3 for traceId=3536004845519906260 spanId=8720177333644726388
2025-11-07 14:43:01.916 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `null`->`null` with priority 0->0 for traceId=3536004845519906260 spanId=1749850157596654480
2025-11-07 14:43:01.917 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpan - Started span: DDSpan [ t_id=3536004845519906260, s_id=1749850157596654480, p_id=8720177333644726388 ] trace=edi-gateway-api/spring.handler/spring.handler tags={_dd.injection.mode=k8s, env=dev, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=0, forceKeep=false, links=[]
2025-11-07 14:43:01.918 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpanContext - setResourceName `null`->`OperationHandler.handle` with priority 0->0 for traceId=3536004845519906260 spanId=1749850157596654480
2025-11-07 14:43:01.921 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpan - Finished span (PENDING): DDSpan [ t_id=3536004845519906260, s_id=1749850157596654480, p_id=8720177333644726388 ] trace=edi-gateway-api/spring.handler/OperationHandler.handle *measured* tags={_dd.injection.mode=k8s, component=spring-web-controller, env=dev, language=jvm, span.kind=server, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=4889105, forceKeep=false, links=[]
2025-11-07 14:43:01.924 [tomcat-handler-60] DEBUG datadog.trace.agent.common.writer.RemoteWriter - Enqueued for serialization: [DDSpan [ t_id=3536004845519906260, s_id=8720177333644726388, p_id=0 ] trace=edi-gateway-api/servlet.request/GET /edi/api/actuator/health *measured* tags={_dd.agent_psr=1.0, _dd.injection.mode=k8s, _dd.profiling.enabled=0, _dd.trace_span_attribute_schema=0, _sample_rate=1, component=tomcat-server, env=dev, http.hostname=100.64.82.138, http.method=GET, http.route=/edi/api/actuator/health, http.status_code=200, http.url=http://100.64.82.138:8088/edi/api/actuator/health, http.useragent=ELB-HealthChecker/2.0, language=jvm, peer.ipv4=172.18.31.109, peer.port=16282, process_id=1, runtime-id=297c889b-aaf3-4e12-a504-b8eb641f364b, servlet.context=/, servlet.path=/edi/api/actuator/health, span.kind=server, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=15676391, forceKeep=false, links=[], DDSpan [ t_id=3536004845519906260, s_id=1749850157596654480, p_id=8720177333644726388 ] trace=edi-gateway-api/spring.handler/OperationHandler.handle *measured* tags={_dd.injection.mode=k8s, component=spring-web-controller, env=dev, language=jvm, span.kind=server, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=4889105, forceKeep=false, links=[]]
2025-11-07 14:43:01.924 [tomcat-handler-60] DEBUG datadog.trace.agent.core.DDSpan - Finished span (WRITTEN): DDSpan [ t_id=3536004845519906260, s_id=8720177333644726388, p_id=0 ] trace=edi-gateway-api/servlet.request/GET /edi/api/actuator/health *measured* tags={_dd.agent_psr=1.0, _dd.injection.mode=k8s, _dd.profiling.enabled=0, _dd.trace_span_attribute_schema=0, _sample_rate=1, component=tomcat-server, env=dev, http.hostname=100.64.82.138, http.method=GET, http.route=/edi/api/actuator/health, http.status_code=200, http.url=http://100.64.82.138:8088/edi/api/actuator/health, http.useragent=ELB-HealthChecker/2.0, language=jvm, peer.ipv4=172.18.31.109, peer.port=16282, process_id=1, runtime-id=297c889b-aaf3-4e12-a504-b8eb641f364b, servlet.context=/, servlet.path=/edi/api/actuator/health, span.kind=server, thread.id=137, thread.name=tomcat-handler-60, version=2.36.0-feature_java_25-SNAPSHOT-2800}, duration_ns=15676391, forceKeep=false, links=[]
2025-11-07 14:43:01.996 [dd-remote-config] DEBUG datadog.communication.ddagent.DDAgentFeaturesDiscovery - discovered traceEndpoint=v0.4/traces, metricsEndpoint=v0.6/stats, supportsDropping=true, supportsLongRunning=true, dataStreamsEndpoint=v0.1/pipeline_stats, configEndpoint=null, evpProxyEndpoint=evp_proxy/v4/, telemetryProxyEndpoint=telemetry/proxy/
2025-11-07 14:43:02.942 [dd-trace-processor] DEBUG datadog.trace.agent.common.writer.ddagent.DDAgentApi - Success while sending 1 (size=1KB) traces. Total: 89, Received: 89, Sent: 89, Failed: 0.
2025-11-07 14:43:02.943 [dd-trace-processor] DEBUG datadog.trace.agent.common.writer.PayloadDispatcherImpl - Successfully sent 1 traces to the API
Expected Behavior
No extra Resources in the APM Service page relating to ignored Resources
Reproduction Code
No response
- Dominant language
- Java
- Stars
- 737
- Forks
- 361
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 173
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/dd-trace-java
-
type: feature request
Difficulty 1/5 1-3 hours Newbie friendliness 70/100
DataDog/dd-trace-java#10245 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
DataDog/dd-trace-java#12608 ·
-
type: bug report
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12597 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12540 · 4 comments · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
DataDog/dd-trace-java#12480 ·
All issues in DataDog/dd-trace-java
Similar issues
-
area/plugin
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
kestra-io/plugin-kestra#190 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
google-ai-edge/LiteRT-LM#3739 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
integra-team-red/meet-map#249 ·
-
[Studio][Bug] Cancelled create-user dialog keeps the password and admin switch for the next attempt Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/rocketmq-dashboard#5064 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
wso2/dpdp-accelerator#287 ·