[Bug] Parallel dedup semi-join planning takes an exclusive lock on gp_segment_configuration
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 53/100
Research direction
Start with the parallel semi-join regression fixture in src/test/regress/sql/cbdb_parallel.sql and trace the two planner paths in src/backend/cdb/cdbpath.c to cdbcomponent_get_maxdbid() in src/backend/cdb/cdbutil.c. Check how the catalog is opened and closed, and identify a read-only lookup that preserves locking for topology-changing callers. Verify with EXPLAIN and the fixture that planning no longer holds an AccessExclusiveLock while topology changes retain their required locking.
Written by the indexing model from the issue text.
Description
Apache Cloudberry version
Apache Cloudberry 2.2.0-incubating, build 13d3c58eaa0 (PostgreSQL 14.10)
What happened
While diagnosing intermittent connection delays on a multi-segment cluster, we observed ordinary read-only SELECTs holding a granted AccessExclusiveLock on pg_catalog.gp_segment_configuration. New connections were waiting for AccessShareLock on the same shared catalog. pg_blocking_pids() identified the SELECT backend as the blocker, including in a different database on the same coordinator.
On a separate three-segment development cluster running the same build, EXPLAIN without ANALYZE alone acquired the lock. In the on/off comparison below, the final plan was Parallel Hash Semi Join in both cases. The lock can therefore come from evaluating a candidate that is never executed.
What you think should happen instead
Planning a read-only query should not hold a transaction-duration AccessExclusiveLock on the shared topology catalog. The planner only needs to read the maximum segment DBID. Please preserve the locking semantics required by topology-changing callers.
How to reproduce
On the affected build in a development environment, use the parallel semi-join regression fixture as a starting point. We compared EXPLAIN (without ANALYZE) on an equivalent EXISTS query joining two distributed tables, with optimizer=off, enable_parallel=on, and two workers per gather.
| Planner settings | Current backend’s lock on gp_segment_configuration after EXPLAIN |
|---|---|
| Both parallel dedup semi-join options enabled | Granted AccessExclusiveLock |
enable_parallel_dedup_semi_join=off and enable_parallel_dedup_semi_reverse_join=off |
No lock observed |
We inspected pg_locks in each read-only test transaction and rolled back immediately after taking the sample.
Operating System
Enterprise Linux 9, x86_64
Anything else
The two parallel join paths in cdbpath.c call cdbcomponent_get_maxdbid() when the deduplicated input uses multiple parallel workers. The catalog implementation opens the relation with AccessExclusiveLock and closes it with NoLock, keeping the lock until transaction end.
A possible fix is to give the planner a separate read-only lookup with appropriately scoped locking. Turning off both dedup options at session scope avoided this path on the tested build while leaving ordinary parallel semi-joins available. We verified the behavior on 2.2.0 only.
Are you willing to submit PR?
- Yes, I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct.
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 273
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 105
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/cloudberry
-
isolation2 lockmodes fails on cassert builds: extra AccessShareLock on partition rootPossibly taken @tuhaihe claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/cloudberry#2356 ·
Maintainers usually reply within 2 days
-
psql tab completion broken for storage servers, tags, profiles and resource groups (pre-PG15 query format)Possibly taken @allenvox claimed this 5 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/cloudberry#2316 ·
Maintainers usually reply within 2 days
-
[Bug] Expand fails to recognize user-defined tablespacesPossibly taken @nix-oss claimed this 92 days ago. Opentype: Bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/cloudberry#1885 · 2 reactions ·
Maintainers usually reply within 2 days
-
gpexpand first stage fails: 'SyncPackages' object has no attribute 'ret' (extension sync never runs; regression)Possibly taken @talmacschen-arch claimed this 153 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
apache/cloudberry#1825 ·
Maintainers usually reply within 2 days
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
apache/cloudberry#2348 · 1 comment ·
Maintainers usually reply within 2 days
All issues in apache/cloudberry
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
windows-artifacts: the win-arm64 job uses a Visual Studio generator missing from windows-11-armOpen
Difficulty 1/5 Under an hour Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
wasm-micro-runtime/wasm-micro-runtime#5217 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
HandBrake/HandBrake#8278 · 1 comment ·
Maintainers usually reply within 1 day