[Bug] 边shard不均导致backup很慢
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- java
- Domain
- cli, databases, performance
Research direction
Start by running the reported bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t edge --directory ./backup_test command and compare edge and vertex backup behavior. Inspect the backup path and shard workload using the thread-activity evidence in the issue; done means edge backup no longer leaves most threads idle and completes faster.
Written by the indexing model from the issue text.
Description
Bug Type (问题类型)
performance (性能下降)
Before submit
- 我已经确认现有的 Issues 与 FAQ 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents)
Environment (环境信息)
- Server Version: 1.0.0 (Apache Release Version)
- Backend: RocksDB x nodes, HDD or SSD
- OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x
- Data Size: xx vertices, xx edges
Expected & Actual behavior (期望与实际表现)
执行边的备份，发现非常慢（点备份很快），命令为：
bin/hugegraph --url http://127.0.0.1:8080 --graph hugegraph backup -t edge --directory ./backup_test
通过源码发现问题应该是边shard不均匀，导致大部分的线程执行完各自的小分片之后退出，最后只会有1～2个线程在执行较大的分片。虽然配置了多线程，但最后只有一个线程在执行备份，其他线程均退出。
也打印了线程池的活跃情况，一开始全部活跃，到最后只有2个活跃。（忽略线程数与备份生成的文件数不一致的情况，修改过线程数）
Vertex/Edge example (问题点 / 边数据举例)
No response
Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
No response
- Dominant language
- Java
- Stars
- 121
- Forks
- 125
- Avg merge
- 8d 14h
- Merged PRs (30d)
- 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/hugegraph-toolchain
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
apache/hugegraph-toolchain#772 ·
-
client enhancement help wanted loader tools
Difficulty 5/5 Over a week Newbie friendliness 25/100
apache/hugegraph-toolchain#748 · 1 comment · 1 reaction ·
-
bug hubble
Difficulty 4/5 3-5 days Newbie friendliness 35/100
apache/hugegraph-toolchain#722 · 1 comment ·
-
apache/hugegraph-toolchain#720 · 1 comment · 1 reaction · 1 assignee ·
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 48/100
apache/hugegraph-toolchain#721 · 3 comments ·
All issues in apache/hugegraph-toolchain
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100