[Bug] Memory leaks occur in HugeGraph Server during data writing.
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start at CachedGraphTransaction.commitMutation2Backend() and trace the notifyChanges(Cache.ACTION_INVALIDED, HugeType.VERTEX, vertexIds) call into the EventHub single-threaded pool. Reproduce with large vertex writes and inspect jmap histograms; done means cache invalidation remains functional without an unbounded backlog retaining vertex ID data.
Written by the indexing model from the issue text.
Description
Bug Type (问题类型)
None
Before submit
- 我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)
Environment (环境信息)
- Server Version: 1.5.0 (Apache Release Version)
- Backend: RocksDB 5 nodes, SSD
Expected & Actual behavior (期望与实际表现)
When memory leaks occur in the graphserver during data writing, the distribution of object quantities in the JVM is as follows:
jmap -histo:live 51680 | head -n 10
num #instances #bytes class name (module)
1: 284880553 13509899520 [B (java.base@11.0.22)
2: 284703909 9110525088 java.lang.String (java.base@11.0.22)
3: 283905229 6813725496 org.apache.hugegraph.backend.id.IdGenerator$StringId
4: 567813 2284841352 [Lorg.apache.hugegraph.backend.id.Id;
5: 1384040 182210368 [Ljava.lang.Object; (java.base@11.0.22)
6: 2270975 90839000 java.util.concurrent.ConcurrentLinkedDeque$Node (java.base@11.0.22)
7: 1191421 76250944 java.util.LinkedHashMap$Entry (java.base@11.0.2
The issue was eventually traced to the CachedGraphTransaction, where there is an action to clear edge caches when writing vertices. If a large number of vertices are written, the commitMutation2Backend() method triggers this.notifyChanges(Cache.ACTION_INVALIDED, HugeType.VERTEX, vertexIds), which results in a backlog of tasks in the single-threaded thread pool within the EventHub, holding onto vertxId data and causing a memory leak.
Vertex/Edge example (问题点 / 边数据举例)
No response
Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
No response
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 637
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 23
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/hugegraph
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 3/5 1-2 days Newbie friendliness 64/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 28/100
All issues in apache/hugegraph
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100