citusdata/citus

Use TRUNCATE + COPY FREEZE when copying a shard

開放

#7,403 建立於 2024年1月5日

 (1 則留言) (0 個反應) (1 位負責人)C (625 個分叉)batch import
good first issueperformance

倉庫指標

星標
 (9,388 顆星)
PR 合併指標
 (平均合併 41天 2小時) (30 天內合併 16 個 PR)

描述

In src/backend/distributed/operations/worker_shard_copy.c file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use the FREEZE option of COPY to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.

What I think is needed to achieve this:

  1. Add the FREEZE option to the COPY command that we generate in ConstructShardCopyStatement
  2. Add the FREEZE option to the list of options created in LocalCopyToShard
  3. Start a transaction in ConnectToRemoteAndStartCopy
  4. Truncate the table in ConnectToRemoteAndStartCopy (before starting the COPY)
  5. End the transaction in ShardCopyDestReceiverShutdown

Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.

貢獻者指南