tikv/tikv

Support caching TiDB schema when processing Copr request

Open

#6,367 创建于 2019年12月29日

在 GitHub 查看
 (0 评论) (0 反应) (0 负责人)Rust (15,922 star) (2,189 fork)batch import
difficulty/hardhelp wantedsig/coprocessor

描述

Description

Currently TiDB always sends all schema TiKV needs in each Coprocessor request. The cost of dealing with the schema may take notable proportion for short OLTP requests.

Considering that DDLs are very few, table schemas are very likely to be unchanged for most of the requests, caching the schema at TiKV side may be helpful for these OLTP requests.

Proposal:

  1. TiDB only sends TableId, TableSchemaUpdateTs, ColumnIds to TiKV in each request.
  2. TiKV checks whether local cached schema is still valid according to TableId and TableSchemaUpdateTs.
  3. If local cached schema is not valid, TiKV returns a SchemaOutdatedError, indicating that TiDB need to re-send full schema to TiKV.
  4. If local cached schema is valid, TiKV uses this schema with supplied ColumnIds to process the request.
  5. TiDB resends the Request carried with full schema to TiKV if the previous request failed with SchemaOutdatedError.

Note: this PCP is still a proposal, which needs some discussion to check details. Remember to sync with mentor if you are interested in this PCP.

Difficulty

  • Hard

Score

  • 4200

Mentor(s)

  • @breeswish
  • @lonng

Recommended Skills

  • Rust and Golang programming

贡献者指南

技术栈
rustgo
领域
databaseperformancebackend
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
5
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
over 1 week
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
mostly clear
前置要求
Rust programmingGolang programmingDistributed systems concepts
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
10
研究方向
This is a proposal to cache TiDB schema on TiKV side for Coprocessor requests. The contributor should first read the full proposal and discuss with mentors @breeswish and @lonng. Key areas to investigate include the current schema handling in TiKV (likely in the coprocessor module), the structure of Coprocessor requests, and how TiDB sends schema information. The implementation would involve adding a schema cache on TiKV, checking validity via TableSchemaUpdateTs, and handling SchemaOutdatedError. Look at existing code in tikv/src/coprocessor/ and related modules.