pingcap/tiflow

[TiCDC] support data compression when syncing to MySQL/TiDB

Aperta

#9733 aperta il 13 set 2023

 (3 commenti) (1 reazione) (0 assegnatari)Go (310 fork)auto 404
help wantedtype/feature

Metriche repository

Star
 (461 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Is your feature request related to a problem?

When deploy multiple TiDB Clusters across multiple regions, TiCDC will be used to replicate data changes between these TiDB Clusters. There are some points need to be considered when doing cross region replication:

  • The network bandwidth of cross regions is limited.
  • The cost of data transfer across regions is higher than data transfer inside a region.

Describe the feature you'd like

Support compression when replicate data changes across regions. MySQL protocol support compression: https://dev.mysql.com/doc/refman/8.0/en/connection-compression-control.html TiDB also support connection compression: https://github.com/pingcap/tidb/issues/22605

Describe alternatives you've considered

  1. Don't support compression, pay more cost for cross region replication; but it is easier to hit cross region network bandwidth limitation when the data replication throughput is large enough.

Teachability, Documentation, Adoption, Migration Strategy

No response

Guida contributor