[Bug] gremlin-sql缓存占用大量内存导致系统卡顿

Open

#2,402 opened on Dec 25, 2023 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 30/100

Issue type: Bug
Clarity: Needs clarification
Activity status: Stale
Tech stack: java
Domain: databases, performance

Research direction

Start at GremlinGroovyClassLoader, focusing on how gremlin-sql scripts are cached after the reported bulk delete request. Reproduce the g_rocksdb.traversal().V(...).drop() workload and inspect heap and GC behavior; done means expired cached script classes are cleared and the workload no longer causes unbounded memory growth.

Written by the indexing model from the issue text.

Description

bug gremlin rocksdb

Bug Type (问题类型)

performance (性能下降)

Before submit

我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)

Environment (环境信息)

Server Version: 1.0.0 (Apache Release Version)
Backend: RocksDB x nodes, HDD or SSD
OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x
Data Size: xx vertices, xx edges

Expected & Actual behavior (期望与实际表现)

在最近一次对数据库的大批量删除操作中，发现了这个问题
使用get请求调用gremlin-sql：g_rocksdb.traversal().V('vertex_id1','vertex_id2','vertex_id3',...).drop()删除了2000w数据后，系统变得十分卡顿。
检查gc日志发现有大量的Allocation Stall

使用mat分析heap-dump发现有大量的Class对象占用了内存

看起来GremlinGroovyClassLoader在管理gremlin-sql的脚本类上边没有设计好缓存的边界

感觉有必要设计一个机制及时清理过期的缓存内容

Vertex/Edge example (问题点 / 边数据举例)

No response

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

No response

Dominant language: Java
Stars: 3.2k
Forks: 637
Avg merge: 3d 18h
Merged PRs (30d): 23

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.