tikv/tikv

Enable rate-bytes-per-sec option by default

Open

#6484 opened on Jan 17, 2020

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Rust (15,922 stars) (2,189 forks)batch import
component/rocksdbcomponent/titandifficulty/mediumhelp wantedsig/engine

Description

Enabling rate-bytes-per-sec would keep rocksdb background compaction steadily runs, smoothing both IO and CPU usage. However few of our users are using the option. Since ansible runs fio test by default, we can use the result to set an initial default value for rate-bytes-per-sec, if user didn't set it.

To be more specific, we can run fio write only with threads = default max-background-jobs, and use the throughput as upper bound. Then we can maybe set rate-bytes-per-sec to 90% of the upper bound. Other suggestions are welcome.

Currently only kvdb respect rate-bytes-per-sec. We should also make raftdb respect it too.

Task breakdown:

  • Make raftdb respect rate-bytes-per-sec
  • Update ansible to set default value for rate-bytes-per-sec
  • benchmark to show what would be a good default rate-bytes-per-sec

Contributor guide

Tech stack
rust
Domain
databaseperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Rust programmingRocksDB configurationAnsible basicsfio benchmarking
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
Start by locating the existing rate bytes per sec usage in kvdb code (likely in src/server or src/raftstore). Then extend raftdb to respect this option by modifying configuration parsing and applying it. For the Ansible component, find the role that sets TiKV configuration (e.g., roles/tikv/templates/tikv.toml.j2) and add logic to compute a default from fio results. Finally, benchmark with fio to determine an appropriate default (e.g., 90% of write throughput). The issue's task list provides clear steps but no assignee or recent activity indicates it requires a fresh start.