elementsd shutdown seems unnecessarily slow for long-running nodes
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start by locating the shutdown path that manually unloads the block index and deallocates its entries, then compare it with the process-exit behavior. Reproduce shutdown after a long-running node if possible, and inspect the test cases that create and destroy multiple block indices. Done means avoiding unnecessary shutdown work while preserving clean deletion where tests require it.
Written by the indexing model from the issue text.
Description
I'm curious if other people are finding that elementsd shutdown takes forever when the node has been running for a very long time, or if this is somehow unique to my machine. It's possible this issue is somehow specific to mac or non-linux platforms; it's also possible it's been made worse by my memory-reducing fix.
What appears to be happening: during shutdown we manually unload the block index, and deallocate the entries, one at a time. This seems basically pointless -- all we're doing is manually freeing memory that will be freed anyway when the process exits. This does not seem to be slow if we've just started, but takes forever when run on a process that's been around for a long time. What I suspect is happening is that the block index has been paged out to disk, and in clearing it we're paging it in, a bit at a time, as we access the entries in random order. Even at SSD speeds, paging in multiple gigabytes using random reads is going to take ages.
(The more problematic possibility is some kind of leak, but I don't see the process having particularly excessive memory usage -- any more than usual -- so I don't think this is it.)
If possible, it would be nice to stop wasting all this effort on shutdown. Assuming it's in fact wasted, one possible catch is the use of the same codepaths in test cases -- we may still need to cleanly delete objects there, if we have to create and destroy multiple block indices before ending the process.
- Dominant language
- C++
- Stars
- 1.2k
- Forks
- 418
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 13
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ElementsProject/elements
-
Difficulty 2/5 1-3 hours Newbie friendliness 58/100
ElementsProject/elements#1555 ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
ElementsProject/elements#1521 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 32/100
ElementsProject/elements#1516 · 5 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
ElementsProject/elements#1502 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 38/100
ElementsProject/elements#1500 · 2 comments ·
All issues in ElementsProject/elements
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
flutter-webrtc/flutter-webrtc#2206 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
google-ai-edge/LiteRT-LM#3739 ·
-
Component: GLib
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
brave/brave-browser#59300 ·
-
Mute ydb/tests/functional/dstool/test_canonical_requests.py.Test.test_group_take_snapshot in main Openai_reviewed
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
ydb-platform/ydb#53974 · 3 comments ·