Event bus: build high-rate topics only for clients that subscribed to them
Maintainers usually reply within 6 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 42/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- rust
- Domain
- backend, distributed-systems
Research direction
Start with crates/blockchain/src/events.rs and crates/net/rpc/src/events.rs, then trace the producer gates and sink trait in ethlambda-network-api; the issue describes two possible designs. Run or add a test where a client subscribes only to head. Done when that subscription does not build single_attestation, and the relevant high-rate topics are handled by the chosen design.
Written by the indexing model from the issue text.
Description
Problem. EventBus::has_subscribers is receiver_count() > 0 (crates/blockchain/src/events.rs:644). Every producer gates on it, so a single SSE client turns on every topic's payload construction, whatever ?topics= it asked for: a validator client that only wants head, for example. The filtering happens afterwards, in the stream (crates/net/rpc/src/events.rs, !topics.contains(&ev.topic())).
For the two high-rate beacon topics that means:
single_attestation(lambdaclass/ethlambda#757) clones every subnet attestation the node accepts: every vote on its backbone subnets, plus the duty subnets it joins.attestationclones every accepted aggregate.
Both go into the one ring all topics share (CHAIN_EVENT_CHANNEL_CAPACITY, 8192, events.rs:600). A client whose connection stalls lags after capacity / total rate, and loses the low-rate events it did ask for, which it learns only from the dropped N messages comment frame. The doc on CHAIN_EVENT_CHANNEL_CAPACITY already names this and calls a per-topic split "the escape hatch". The rates on a mainnet node are not measured; lean_-prefixed counters per topic emitted would measure them.
Why not in lambdaclass/ethlambda#757. 757 adds one producer and follows the bus's existing gate, as attestation already does. Per-topic interest changes the bus, the SSE handler and every producer's gate.
Fix. Either of two:
- Count subscriptions per
Topic. The SSE handler registers its topic set with a guard that decrements on drop, andEventBus::wants(Topic)replaceshas_subscribersat each producer; the sink trait inethlambda-network-apigrows the same method. - Keep
has_subscribers, but givesingle_attestationandattestationa ring of their own, so their rate cannot push low-rate events out of a slow client's window.
A test: a client subscribed to head alone causes no single_attestation to be built.
Also high-rate (from lambdaclass/ethlambda#748): the gloas payload_attestation_message and execution_payload_bid events are built per message too, so the same per-topic interest check should cover them.
- Dominant language
- Rust
- Stars
- 86
- Forks
- 34
- Avg merge
- 9d 20h
- Merged PRs (30d)
- 77
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from lambdaclass/ethlambda
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
lambdaclass/ethlambda#850 ·
Maintainers usually reply within 6 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
lambdaclass/ethlambda#777 · 1 reaction ·
Maintainers usually reply within 6 days
-
Run all gossip validation testsMay be free again @dicethedev claimed this 9 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
lambdaclass/ethlambda#725 · 2 comments ·
Maintainers usually reply within 6 days
-
Handle malformed DB dataMay be free again @dicethedev claimed this 115 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
lambdaclass/ethlambda#529 · 4 comments ·
Maintainers usually reply within 6 days
-
Document architectural choicesMay be free again @dicethedev claimed this 115 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
lambdaclass/ethlambda#521 · 2 comments ·
Maintainers usually reply within 6 days
All issues in lambdaclass/ethlambda
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
antithesishq/bombadil#473 ·
Maintainers usually reply within 2 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
libracoder/kiri#669 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 5 days
-
autofixer: follow-up
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
tursodatabase/turso#12217 ·
Maintainers usually reply within 1 day
-
Declare the CMS schema library in the optional recipients extraPossibly taken A pull request linked to this issue is open or already merged. Openarea:python in-progress
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
yhay81/pyropdf#1005 · 2 comments ·
Maintainers usually reply within 1 day