Limit unbounded channel usage

Open
#5,855 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Stale
Tech stack
rust

Research direction

Search Forest for every flume::unbounded usage, beginning with the peer-to-peer path around handle_gossip_event, and review how each channel is used when it reaches its limit. Check the clippy settings as well. Done means every usage is justified or changed to a bounded channel with a sane constant or parameter, and unbounded usage is banned in clippy without introducing deadlocks.

Written by the indexing model from the issue text.

Description

Type: Task

Summary

During recent memory profiling in Forest, we noticed the channels in the peer-to-peer module tend to be unbounded. This could have led to excessive memory usage.

Image

A likely scenario here is that messages in handle_gossip_event get deserialised and put into the network channel. Get enough of such events and the channels could grow quickly, each holding the deserialised message, leading to high memory usage.

This issue is not exclusive to the p2p layer; we should revisit all of the flume::unbounded in Forest and allow it only when it's justified.

Completion Criteria

  • re-visit flume::unbounded usages in Forest; all of them should be either justified or changed to bounded versions, with either constants or parameters with sane defaults. ⚠️ Important! Usage of these channels should also be revisited - pushing to a channel that is at its limits will block; this could lead to a deadlock.
  • ban the usage of flume::unbounded in the clippy settings

Additional Links & Resources

Dominant language
Rust
Stars
700
Forks
201
Avg merge
1d 17h
Merged PRs (30d)
65

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ChainSafe/forest

All issues in ChainSafe/forest

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.