yugabyte/yugabyte-db

[DocDB] Allow Docdb flush/compact rate limiter to specify the granularity of enforcement.

Open

#13,404 创建于 2022年7月22日

在 GitHub 查看
 (2 评论) (0 反应) (1 负责人)C (1,003 fork)batch import
area/docdbgood first issuekind/enhancementpriority/medium

仓库指标

Star
 (8,229 star)
PR 合并指标
 (平均合并 17天 21小时) (30 天内合并 92 个 PR)

描述

Jira Link: DB-3034

Description

Rocksdb Rate limiter supports specifying the granularity at which the rate is enforced. This defaults to 100ms

extern RateLimiter* NewGenericRateLimiter(
    int64_t rate_bytes_per_sec,
    int64_t refill_period_us = 100 * 1000,
    int32_t fairness = 10);

However, Docdb doesn't seem to be specifying the granularity and only uses the 100ms default. This can be an issue if the underlying cloud provider enforces their rate limits at a smaller interval.

std::shared_ptr<rocksdb::RateLimiter> CreateRocksDBRateLimiter() {
  if (PREDICT_TRUE((FLAGS_rocksdb_compact_flush_rate_limit_bytes_per_sec > 0))) {
    return std::shared_ptr<rocksdb::RateLimiter>(
      rocksdb::NewGenericRateLimiter(FLAGS_rocksdb_compact_flush_rate_limit_bytes_per_sec));
  }
  return nullptr;
}

A large chunk of write from a flush/compaction may adversely impact our latencies on WAL writes.

贡献者指南