[Improvement] Improve the current SQL of JDBCBackend
#4 024 ouverte le 2 juil. 2024
Métriques du dépôt
- Stars
- (3 058 stars)
- Métriques de merge PR
- (Métriques PR en attente)
Description
What would you like to be improved?
In the current implementation of xxxMetaMapper and xxxMetaService, we seldom use JOIN and transactions, for example:
The list catalog operation will issue two SQL queries, the first one is to get a metalake id, then using this id to get the catalog list.
Instead of issuing two queries, we can use JOIN to join catalog and metalake table to get catalog list by one query, this will save the IO time and avoid inconsistent problem.
Also for updateCatalog
We have several steps:
- get metalake id.
- get catalog PO.
- update the catalog object.
- update the catalog in RDBMS.
These 4 steps are not in the transaction, which will potentially meet the inconsistent problem, and we highly leverage the lock to avoid the concurrent problem. A better solution is to put these 4 steps into a transaction.
How should we improve?
So basically, we can improve the current SQL to:
- reduce the queries and increase the performance.
- Put multiple step actions into one transaction to leverage DB's transaction to keep consistent.