JVM SIGSEGV crashes after upgrade from 1.61.1 to 1.62.0
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- java
- Domain
- devtools, performance
Research direction
Start by reviewing the attached logs.txt and the crash entry points named in the report, especially JavaProfiler.dump0, AgentTaskScheduler, DefaultConfigurationPoller, and the shaded okhttp path. Compare changes between 1.61.1 and 1.62.0, then reproduce on Java 25; done means identifying the regression and confirming the JVM no longer crashes.
Written by the indexing model from the issue text.
Description
Tracer Version(s)
1.62.0
Java Version(s)
25
JVM Vendor
Eclipse Adoptium / Temurin
Bug Report
After upgrading dd-trace-java from 1.61.1 to 1.62.0, the JVM started crashing with SIGSEGV (and one UNKNOWN signal) on multiple tasks.
Reverting to 1.61.1 with no other changes made the crashes stop.
Crash tracking is enabled, and we have collected several distinct crash signatures. They cluster into three groups:
-
SIGSEGV inside the Datadog profiler native dump path:
Profiler::updateThreadName(jvmtiEnv*, JNIEnv, _jobject, bool)
-> jvmti_Deallocate
called from JavaProfiler.dump0 via the profiling system's periodic
snapshot task (AgentTaskScheduler$PeriodicTask). -
SIGSEGV in JIT inline-cache / wrong-method handling:
SharedRuntime::find_callee_info_helper
-> handle_ic_miss_helper / handle_wrong_method
Observed on multiple call sites, including:- the agent's own shaded okhttp during remote-config polling
(datadog.okhttp3...Http1Codec$FixedLengthSink.close, called from
DefaultConfigurationPoller.fetchConfiguration), seen repeatedly - a Micrometer StepMeterRegistry rollover scheduled task
(io.micrometer.core.instrument.step.StepTuple2.rollCount)
- the agent's own shaded okhttp during remote-config polling
-
JVM-internal crashes around the code cache / GC:
- nmethod::is_unloading -> DependencyContext::clean_unloading_dependents
on a G1 code-cache unloading worker - G1CodeRootSet::add(nmethod*) -> nmethod::oops_do -> register_nmethod
on a C1 CompileBroker thread installing a freshly compiled nmethod
(crashed inside concurrentHashTable.inline.hpp:684)
- nmethod::is_unloading -> DependencyContext::clean_unloading_dependents
All crashes occur either on a Datadog-managed thread or in code installed by the agent (its shaded okhttp). The stacks above come from dd-trace-java's own crashtracking output, which was enabled at the time.
hs_err_pid*.log files are not available: this is running on AWS ECS, and the crashing tasks were recycled before we could retrieve them.
Workaround: pinning back to 1.61.1 removes the crashes entirely.
Expected Behavior
Upgrading dd-trace-java from 1.61.1 to 1.62.0 should not cause JVM crashes.
Reproduction Code
No response
- Dominant language
- Java
- Stars
- 737
- Forks
- 361
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 173
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/dd-trace-java
-
type: feature request
Difficulty 1/5 1-3 hours Newbie friendliness 70/100
DataDog/dd-trace-java#10245 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
DataDog/dd-trace-java#12608 ·
-
type: bug report
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12597 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12540 · 4 comments · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
DataDog/dd-trace-java#12480 ·
All issues in DataDog/dd-trace-java
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
inu-appcenter/memorIN-backend#288 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
frontend maui-pilot pilot-ask question
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
area/plugin
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
kestra-io/plugin-kestra#190 ·