SIGSEGV crash in Dictionary::lookup when profiling with Pyroscope
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- java, kubernetes, linux
- Domain
- observability, performance
Research direction
Start at the native Dictionary::lookup entry point named in the crash and review the async-profiler path used when CPU, ALLOC, and LOCK profiling run together. Reproduce the configuration in the reported Linux container environment and compare the fatal error details across affected instances. Done means identifying and preventing the SIGSEGV in this profiling combination.
Written by the indexing model from the issue text.
Description
Hello:
We are experiencing intermittent JVM crashes (SIGSEGV) on a small number of container instances when using async-profiler via Pyroscope. The crash occurs in the native Dictionary::lookup function.
Most containers run fine — only a few pods are affected, which suggests a possible race condition or memory corruption issue.
How Pyroscope is started:
We enable CPU, ALLOC, and LOCK profiling simultaneously via the Pyroscope Java agent:
java -javaagent:pyroscope.jar
-Dpyroscope.application.name=***
-Dpyroscope.server.address=***
-Dpyroscope.profiler.event=CPU
-Dpyroscope.profiler.alloc=4096k
-Dpyroscope.profiler.lock=20ms
-jar app.jar
Environment:
OS: Linux (amd64), containerized (Kubernetes)
- JRE: Java(TM) SE Runtime Environment 17.0.10+11-LTS-240
- JVM: Java HotSpot(TM) 64-Bit Server VM (mixed mode, sharing, tiered, compressed oops, compressed class ptrs, G1 GC)
- Pyroscope version: 2.9.1
- Container base image: <e.g., oracle/graalvm-ce:17, eclipse-temurin:17, etc.>
Crash details:
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007f3bf4e08965, pid=1, tid=596
JRE version: Java(TM) SE Runtime Environment (17.0.10+11) (build 17.0.10+11-LTS-240)
Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0.10+11-LTS-240, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
Problematic frame:
C [libasyncProfiler-linux-x64-94af16d95b27e7e3346a51e8d43e744de92a62a3.so+0xd965] Dictionary::lookup(char const*, unsigned long)+0x65
An error report file with more information is saved as:
If you would like to submit a bug report, please visit:
https://bugreport.java.com/bugreport/crash.jsp
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f44abbdabc7]
- Dominant language
- Java
- Stars
- 125
- Forks
- 48
- Avg merge
- 1h 8m
- Merged PRs (30d)
- 1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from grafana/pyroscope-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
grafana/pyroscope-java#294 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
grafana/pyroscope-java#360 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
grafana/pyroscope-java#283 ·
-
grafana/pyroscope-java#250 · 1 comment · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
grafana/pyroscope-java#237 · 2 comments ·
All issues in grafana/pyroscope-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100