Missing Drools classloader in `canSkipClassLoaderByName` skip-list: `DynamicProjectClassLoader$DefaultInternalTypesClassLoader` causes severe startup/reload slowdown with the Drools executable-model canonical KieModule
@amarziali is already working on this.
Since Sep 11, 2026.
Assessment
This issue has not been assessed yet.
Description
Tracer Version(s)
1.66.0
Java Version(s)
25.0.3
JVM Vendor
Amazon Corretto
Bug Report
ClassLoaderMatchers.canSkipClassLoaderByName() already hardcodes a skip for
three Drools classloaders:
// drools
case "org.drools.core.rule.PackageClassLoader":
case "org.drools.wiring.dynamic.PackageClassLoader":
case "org.drools.core.rule.JavaDialectRuntimeData$PackageClassLoader":
return true;
A fourth Drools-internal classloader used by the newer executable-model
canonical KieModule build path is not in that list:
org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader
Because it's missing, every class defined through this classloader falls
through into the full instrumentation-matching pipeline, including
ClassLoaderMatchers.buildHasClassMask(), which does an uncached
getResource() scan per class-name check across every active
instrumentation module. Drools creates a fresh instance of this
classloader for every KieBase build (once per rule, via
CanonicalKieModule.createKieBase → ProjectClassLoader.loadClass →
defineType), so the ClassLoaderValue cache backing hasClassMask()
never gets a hit — the expensive scan runs from scratch for every single
rule, every time a KieBase is (re)built (startup load of N rules, or any
runtime rule reload after a cache eviction).
At ~2000 rules this is severe: a single-threaded rule-load thread showed
~99% CPU time (elapsed=99.00s, cpu=76112.16ms) stuck entirely in this
path (full stack under Reproduction Code below).
Rule-load time with 8 concurrent loader threads did not improve at all vs.
1 thread (~264s either way) — evidence this was a per-classloader cache-miss
cost, not lock contention, since threads only ever contended on the
symptom (UrlJarFiles$Cache / URLClassPath locks under classloading),
not the root cause. Environment: Drools 8.44.2.Final, executable-model /
canonical KieModule path, ~2000 rules loaded via KieRepositoryImpl.addKieModule
→ CanonicalKieModule.createKieBase.
Confirmed fix / workaround: adding the missing classloader name to
dd.trace.classloaders.exclude / DD_TRACE_CLASSLOADERS_EXCLUDE (the
user-facing config already backed by the same skip-list check) resolves it
completely:
DD_TRACE_CLASSLOADERS_EXCLUDE=org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader
Before / after, same rule set, same machine:
| Config | Rule-load time |
|---|---|
| 8 threads, no exclude | ~264,000 ms |
| 1 thread, no exclude | did not complete in 8+ min (unbounded — cost scales per rule) |
| 1 thread, with exclude | ~193,000 ms |
| 8 threads, with exclude | ~54,000 ms |
After the exclude, the same thread's stack shows it spending its time in
legitimate work only (S3 fetch + zip decompression of the rule's KJar), with
no Datadog instrumentation-matching frames at all.
Ask: please add
org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader
to the existing Drools block in canSkipClassLoaderByName
(dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/bytebuddy/matcher/ClassLoaderMatchers.java),
alongside the three classloaders already there, so this becomes the default
instead of requiring the manual config workaround:
// drools
case "org.drools.core.rule.PackageClassLoader":
case "org.drools.wiring.dynamic.PackageClassLoader":
case "org.drools.core.rule.JavaDialectRuntimeData$PackageClassLoader":
case "org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader": // add this
return true;
Happy to open a PR with this one-line addition if that's preferred over a
maintainer making the change.
Expected Behavior
ClassLoaderMatchers.canSkipClassLoaderByName() should treat
org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader
the same as the three Drools classloaders already hardcoded in that method
(org.drools.core.rule.PackageClassLoader,
org.drools.wiring.dynamic.PackageClassLoader,
org.drools.core.rule.JavaDialectRuntimeData$PackageClassLoader) — i.e.
return true and skip the class immediately, without ever reaching
buildHasClassMask()'s getResource() scan.
Since Drools creates a fresh instance of this classloader per KieBase build,
the expected outcome is that repeated KieBase construction (startup rule
loading, or any runtime rule reload after a cache eviction) does not pay a
per-classloader, per-instrumentation-module getResource() cost on every
single build. Setting DD_TRACE_CLASSLOADERS_EXCLUDE to this classloader
name manually reproduces this expected behavior today (264s → 54s for ~2000
rules in our environment) — the ask is simply to make that the default,
matching how the other three Drools classloaders are already handled.
Reproduction Code
No isolated standalone reproducer (this surfaces at scale inside a large
Drools application, ~2000 rules), but the mechanism is straightforward to
reproduce in principle:
// with dd-java-agent attached, no DD_TRACE_CLASSLOADERS_EXCLUDE set:
for (int i = 0; i < 2000; i++) {
// build a fresh executable-model KieBase per rule/module, e.g.:
KieServices ks = KieServices.get();
KieContainer kc = ks.newKieContainer(releaseId, ProjectClassLoader.class.getClassLoader());
KieBase kb = kc.getKieBase(kBaseName); // triggers CanonicalKieModule.createKieBase
}
Each iteration creates a new DynamicProjectClassLoader$DefaultInternalTypesClassLoader
instance; with the agent attached and this classloader missing from the
skip-list, thread-dump sampling during the loop shows threads stuck in:
at java.lang.ClassLoader.getResource(...)
at org.drools.wiring.api.classloader.ProjectClassLoader.getResource(ProjectClassLoader.java:309)
at org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader.getResource(DynamicProjectClassLoader.java:145)
at datadog.trace.agent.tooling.bytebuddy.matcher.ClassLoaderMatchers.buildHasClassMask(ClassLoaderMatchers.java:175)
at datadog.trace.agent.tooling.bytebuddy.matcher.ClassLoaderMatchers$1.computeValue(ClassLoaderMatchers.java:150)
at datadog.instrument.utils.ClassLoaderValue.computeValueForKey(ClassLoaderValue.java:185)
at datadog.instrument.utils.ClassLoaderValue.getOtherValue(ClassLoaderValue.java:193)
at datadog.instrument.utils.ClassLoaderValue.get(ClassLoaderValue.java:92)
at datadog.trace.agent.tooling.bytebuddy.matcher.ClassLoaderMatchers.hasClassMask(ClassLoaderMatchers.java:160)
at datadog.trace.agent.tooling.bytebuddy.matcher.ClassLoaderMatchers.incompatibleClassLoader(ClassLoaderMatchers.java:70)
at datadog.trace.agent.tooling.bytebuddy.matcher.GlobalIgnoresMatcher.matches(GlobalIgnoresMatcher.java:39)
at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:12799)
...
at org.drools.wiring.api.classloader.ProjectClassLoader.tryDefineType(ProjectClassLoader.java:193)
at org.drools.wiring.api.classloader.ProjectClassLoader.loadClass(ProjectClassLoader.java:130)
at org.drools.modelcompiler.CanonicalKieModule.createInstance(CanonicalKieModule.java:175)
at org.drools.modelcompiler.CanonicalKieModule.createKieBase(CanonicalKieModule.java:243)
Adding DD_TRACE_CLASSLOADERS_EXCLUDE=org.drools.wiring.dynamic.DynamicProjectClassLoader$DefaultInternalTypesClassLoader
removes this stack from every subsequent iteration.
- Dominant language
- Java
- Stars
- 737
- Forks
- 361
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 173
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/dd-trace-java
-
type: feature request
Difficulty 1/5 1-3 hours Newbie friendliness 70/100
DataDog/dd-trace-java#10245 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
DataDog/dd-trace-java#12608 ·
-
type: bug report
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12597 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12540 · 4 comments · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
DataDog/dd-trace-java#12480 ·
All issues in DataDog/dd-trace-java
Similar issues
-
certification
Difficulty 1/5 Under an hour Newbie friendliness 80/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
[BUG] ECR GetAuthorizationToken returns a proxyEndpoint for the default region, not the request's Openbug ecr
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Needs: Triage Type: Feature request
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
AntennaPod/AntennaPod#8794 ·
-
agentic-workflows
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
github/copilot-sdk#2760 ·