ipfs/kubo

Implement bandwidth limiting

Open

#3,065 opened on Aug 9, 2016

View on GitHub
 (76 comments) (73 reactions) (0 assignees)Go (13,906 stars) (2,725 forks)batch import
exp/experthelp wantedstatus/deferredtopic/libp2p

Description

We need to place limits on the bandwidth ipfs uses. We can do this a few different ways (or a combination thereof):

  • per peer limiting on each actual connection object
    • pros:
      • low coordination cost (no shared objects between connections)
      • should have lower impact on performance than blindly rate limiting the whole process
    • cons:
      • no flow control between protocols, dht could drown out bitswap traffic
  • per subnet limiting
    • pros:
      • avoids rate-limiting LAN/localhost connections.
    • cons:
      • it's not always possible to tell what's "local" (e.g., with IPv6).
  • per protocol limiting on each stream
    • pros:
      • should have the lowest impact on system performance of the three options
      • each protocol gets its own slice of the pie and doesnt impact others
    • cons:
      • increased coordination required, need to reference the same limits across multiple streams
      • still makes it difficult to precisely limit the overall bandwidth usage.
  • global limiting using a single rate limiter over all connections
    • pros:
      • will successfully limit the amount of bandwidth ipfs uses.
    • cons:
      • ipfs will be quite slow when rate limited in this way

Related Issues:

Contributor guide

Tech stack
go
Domain
backendperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Go programmingUnderstanding of IPFS architectureKnowledge of rate limiting concepts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
The issue proposes multiple approaches to bandwidth limiting (per peer, per subnet, per protocol, global). Start by reviewing the pros/cons outlined in the body and the linked issues (e.g., #2489, #2828). Look at the existing connection and stream handling code in go libp2p to understand where rate limiters could be integrated. Also check if there are any pending PRs or recent discussions in the comments that indicate preferred direction. Focus on the simplicity and impact of each approach before proposing an implementation.
Implement bandwidth limiting · ipfs/kubo#3065 | Good First Issue