Deadlock: DynamicCodeGenerated allocates under _stubs_lock while the sampling signal handler spins in findRuntimeStub

Start with ddprof-lib/src/main/cpp/hotspot/jitCodeCache.cpp, focusing on DynamicCodeGenerated and findRuntimeStub; the issue describes the lock/allocation cycle and two possible fix directions. Reproduce or inspect the relevant locking and allocation paths, then add or run tests covering signal-handler lookup while the stub lock is held. Done means the deadlock is avoided without unbounded spinning; the issue names no specific test file.

Summary

The native profiler (ddprof) deadlocks the JVM. A thread that holds the JitCodeCache stub lock calls malloc, while the profiler sampling signal interrupts another thread inside malloc, and that signal handler waits for the stub lock. All other sampled threads then spin in JitCodeCache::findRuntimeStub at 100% CPU, and the JVM never reaches a safepoint (jcmd cannot attach, the service stops answering).

We saw it twice in 4 hours on 2026-10-07, on 2 hosts, with the same cycle each time. We had seen the same symptom (N cores at 100%, JVM unresponsive, no log line) every few hours on this service before we captured stacks, on x86_64 and on arm64 (Graviton) hosts. Both captures below are from x86_64 hosts.

Versions

• dd-java-agent 1.65.1, which loads ddprof 1.49.0 (libjavaProfiler)
• Profiling on (-Ddd.profiling.enabled=true), default profiler settings (CPU and wall-clock samplers)
• Amazon Corretto 25.0.0+36 (JDK 25), x86_64, G1 GC
• Amazon Linux 2023, glibc malloc, kernel 6.1
• The application calls a native library (libvips, through glib) from many threads, so threads are often inside malloc.

JitCodeCache::DynamicCodeGenerated and JitCodeCache::findRuntimeStub in ddprof-lib/src/main/cpp/hotspot/jitCodeCache.cpp are the same in v_1.49.0, v_1.51.0 and main (checked on 2026-10-07), so we think the newer versions are affected too.

The cycle

DynamicCodeGenerated calls _runtime_stubs.add() (which allocates) while it holds _stubs_lock, and findRuntimeStub takes the same lock from the signal handler with the unbounded lockShared(). Possible fixes: do not allocate under _stubs_lock (prepare the entry first), or use the bounded tryLockShared(max_spins) in findRuntimeStub when it runs in a signal handler and give up the frame.

Stacks, capture 1 (6 threads at 100% CPU, 5 of them in findRuntimeStub)

Collected with gdb -p <pid> -batch -ex "thread apply all bt". Addresses and paths removed.

Thread "Service Thread":
#0  __lll_lock_wait_private ()
#1  malloc ()
#2  CodeCache::add(void const*, int, char const*, bool) ()
#3  JitCodeCache::DynamicCodeGenerated(_jvmtiEnv*, char const*, void const*, int) ()
#4  JvmtiExport::post_dynamic_code_generated_internal(char const*, void const*, void const*) ()
#5  JvmtiDeferredEvent::post() ()
#6  ServiceThread::service_thread_entry(JavaThread*, JavaThread*) ()
#7  JavaThread::thread_main_inner() [clone .part.0] ()
#8  Thread::call_run() ()
#9  thread_native_entry(Thread*) ()
#10 start_thread ()
#11 clone3 ()

Thread "<HTTP worker thread>":
#0  JitCodeCache::findRuntimeStub(void const*) ()
#1  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, void const*, unsigned long, unsigned long, int, bool*) ()
#2  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, int, bool*) ()
#3  HotspotSupport::walkJavaStack(StackWalkRequest&) ()
#4  Profiler::recordSample(void*, unsigned long long, int, int, unsigned long long, Event*, unsigned long long*) ()
#5  CTimer::signalHandler(int, siginfo_t*, void*) ()
#6  <signal handler called> ()
#7  malloc_consolidate ()
#8  _int_malloc ()
#9  calloc ()
#10 g_malloc0 ()
#11 g_type_create_instance ()
#12 g_object_new_internal ()
#13 g_object_new_with_properties ()
   ...

Thread "<HTTP worker thread>":
#0  JitCodeCache::findRuntimeStub(void const*) ()
#1  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, void const*, unsigned long, unsigned long, int, bool*) ()
#2  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, int, bool*) ()
#3  HotspotSupport::walkJavaStack(StackWalkRequest&) ()
#4  Profiler::recordSample(void*, unsigned long long, int, int, unsigned long long, Event*, unsigned long long*) ()
#5  CTimer::signalHandler(int, siginfo_t*, void*) ()
#6  <signal handler called> ()
#7  hwy::x86::Cpuid (abcd=..., count=..., level=...)
#8  hwy::x86::FlagsFromCPUID ()
   ...

Stacks, capture 2 (8 threads at 100% CPU, all 8 in findRuntimeStub)

Thread "Service Thread":
#0  __lll_lock_wait_private ()
#1  malloc ()
#2  CodeCache::add(void const*, int, char const*, bool) ()
#3  JitCodeCache::DynamicCodeGenerated(_jvmtiEnv*, char const*, void const*, int) ()
#4  JvmtiExport::post_dynamic_code_generated_internal(char const*, void const*, void const*) ()
#5  JvmtiDeferredEvent::post() ()
#6  ServiceThread::service_thread_entry(JavaThread*, JavaThread*) ()
#7  JavaThread::thread_main_inner() [clone .part.0] ()
#8  Thread::call_run() ()
#9  thread_native_entry(Thread*) ()
#10 start_thread ()
#11 clone3 ()

Thread "<HTTP worker thread>":
#0  JitCodeCache::findRuntimeStub(void const*) ()
#1  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, void const*, unsigned long, unsigned long, int, bool*) ()
#2  HotspotSupport::walkVM(void*, _asgct_callframe*, int, StackWalkFeatures, EventType, int, bool*) ()
#3  HotspotSupport::walkJavaStack(StackWalkRequest&) ()
#4  Profiler::recordSample(void*, unsigned long long, int, int, unsigned long long, Event*, unsigned long long*) ()
#5  CTimer::signalHandler(int, siginfo_t*, void*) ()
#6  <signal handler called> ()
#7  _int_malloc ()
#8  calloc ()
#9  g_malloc0 ()
#10 g_closure_new_simple ()
#11 g_cclosure_new ()
#12 g_signal_connect_data ()
#13 ??? ()
   ...

top -H on both hosts showed the worker threads at 99.9% CPU each, while the VM Thread was sleeping (state S, under 4% CPU). jcmd <pid> Thread.print failed with AttachNotSupportedException ... doesn't respond within 10500ms.

We worked around it with DD_PROFILING_DDPROF_ENABLED=false (JFR-based profiling). We can test a fix build if useful.