apache/gravitino

[Improvement] Improve the current SQL of JDBCBackend

Open

#4,024 创建于 2024年7月2日

在 GitHub 查看
 (6 评论) (0 反应) (0 负责人)Java (887 fork)auto 404
help wantedimprovement

仓库指标

Star
 (3,058 star)
PR 合并指标
 (PR 指标待抓取)

描述

What would you like to be improved?

In the current implementation of xxxMetaMapper and xxxMetaService, we seldom use JOIN and transactions, for example:

The list catalog operation will issue two SQL queries, the first one is to get a metalake id, then using this id to get the catalog list.

Instead of issuing two queries, we can use JOIN to join catalog and metalake table to get catalog list by one query, this will save the IO time and avoid inconsistent problem.

Also for updateCatalog

We have several steps:

  1. get metalake id.
  2. get catalog PO.
  3. update the catalog object.
  4. update the catalog in RDBMS.

These 4 steps are not in the transaction, which will potentially meet the inconsistent problem, and we highly leverage the lock to avoid the concurrent problem. A better solution is to put these 4 steps into a transaction.

How should we improve?

So basically, we can improve the current SQL to:

  1. reduce the queries and increase the performance.
  2. Put multiple step actions into one transaction to leverage DB's transaction to keep consistent.

贡献者指南