ipfs/kubo

Block cache lock contention

Open

#6,934 opened on Feb 26, 2020

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Go (2,725 forks)batch import
exp/experthelp wantedkind/enhancementstatus/readytopic/perf

Repository metrics

Stars
 (13,906 stars)
PR merge metrics
 (Avg merge 4d 2h) (28 merged PRs in 30d)

Description

There is quite a bit of lock contention in the block cache. On the gateways, out of 32 walk threads:

  • 13 were waiting on the disk.
  • 19 were blocked on the cache.
    • 10 trying to add blocks
    • 9 trying to get them.

We're currently using github.com/hashicorp/golang-lru.

Contributor guide

Research direction
Investigate lock contention in the block cache by analyzing the current use of github.com/hashicorp/golang lru and propose a solution such as using a sharded cache or replacing with a more concurrent friendly implementation.
Tech stack
go
Domain
backend
Issue type
Performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30