[Bug] DROP COLUMN of a distribution key after ROLLBACK TO SAVEPOINT leaves the table distributed by a dropped column

Start with set_random_distribution_if_drop_distkey() in tablecmds.c and the duplicate logic in RemoveAttributeById() in heap.c; trace how policy changes interact with relcache invalidation and subtransaction rollback. Add the described isolation2 regression spec and verify it covers both append-optimized and heap tables. Done means retrying DROP COLUMN after ROLLBACK TO SAVEPOINT sets random distribution, and both tables remain readable.

Apache Cloudberry version

main @ 845fdeccd06 (2026-10-27). The code involved (set_random_distribution_if_drop_distkey() in tablecmds.c and the duplicate block in RemoveAttributeById() in heap.c) has not changed since the initial Cloudberry code dump, so older releases are affected as well.

What happened

Inside a subtransaction, if ALTER TABLE ... DROP COLUMN <distribution key> fails partway through and is rolled back with ROLLBACK TO SAVEPOINT, retrying the same DROP COLUMN in that transaction no longer switches the table to random distribution. After COMMIT, gp_distribution_policy.distkey still points at the dropped column:

\d t
...
Distributed by: ("........pg.dropped.1........" int4_ops)

The table is then unusable. Every read and write fails:

SELECT count(*) FROM t;
ERROR:  could not find hash function for type 0 in operator family 1977 (cdbhash.c:449)
INSERT INTO t VALUES (100);
ERROR:  could not find hash function for type 0 in operator family 1977 (cdbhash.c:449)

The retried ALTER TABLE also skips the usual NOTICE: dropping a column that is part of the distribution policy forces a NULL distribution policy, which shows it never took the "make it random" branch.

Both heap and append-optimized tables are affected.

What you think should happen instead

The retried DROP COLUMN should behave exactly as it does when nothing failed before it: emit the NOTICE, set the policy to DISTRIBUTED RANDOMLY, and leave a table that can still be read and written.

How to reproduce

You only need SQL and two sessions; no debugger or fault injector. Session 2 holds a lock on gp_distribution_policy, so session 1's ALTER TABLE times out inside set_random_distribution_if_drop_distkey(). That happens after rd_cdbpolicy has been modified in place and before the catalog is updated.

isolation2 spec (passes pg_isolation2_regress as written):

CREATE TABLE drop_distkey_ao (a int, b int) WITH (appendonly=true) DISTRIBUTED BY (a);
CREATE TABLE drop_distkey_heap (a int, b int) DISTRIBUTED BY (a);
INSERT INTO drop_distkey_ao SELECT i, i FROM generate_series(1, 10) i;
INSERT INTO drop_distkey_heap SELECT i, i FROM generate_series(1, 10) i;

-- Load both relcache entries in session 1 before the catalog is locked.
1: SELECT count(*) FROM drop_distkey_ao;
1: SELECT count(*) FROM drop_distkey_heap;

1: BEGIN;
1: SAVEPOINT s1;
2: BEGIN;
2: LOCK TABLE gp_distribution_policy IN ACCESS EXCLUSIVE MODE;
1: SET LOCAL lock_timeout = '1s';
1: ALTER TABLE drop_distkey_ao DROP COLUMN a;     -- ERROR: canceling statement due to lock timeout
1: ROLLBACK TO s1;
1: SAVEPOINT s2;
1: SET LOCAL lock_timeout = '1s';
1: ALTER TABLE drop_distkey_heap DROP COLUMN a;   -- ERROR: canceling statement due to lock timeout
1: ROLLBACK TO s2;
2: COMMIT;

-- Catalog is still correct here: distkey = 1 for both.
1: SELECT localoid::regclass, distkey FROM gp_distribution_policy
   WHERE localoid IN ('drop_distkey_ao'::regclass, 'drop_distkey_heap'::regclass) ORDER BY 1;

1: ALTER TABLE drop_distkey_ao DROP COLUMN a;     -- no NOTICE
1: ALTER TABLE drop_distkey_heap DROP COLUMN a;   -- no NOTICE
1: COMMIT;

SELECT localoid::regclass, distkey FROM gp_distribution_policy
WHERE localoid IN ('drop_distkey_ao'::regclass, 'drop_distkey_heap'::regclass) ORDER BY 1;
SELECT count(*) FROM drop_distkey_ao;
SELECT count(*) FROM drop_distkey_heap;

Actual output of the last part:

 localoid          | distkey
-------------------+---------
 drop_distkey_ao   | 1
 drop_distkey_heap | 1
(2 rows)
SELECT count(*) FROM drop_distkey_ao;
ERROR:  could not find hash function for type 0 in operator family 1977 (cdbhash.c:449)
SELECT count(*) FROM drop_distkey_heap;
ERROR:  could not find hash function for type 0 in operator family 1977 (cdbhash.c:449)

Expected: distkey empty for both tables, and count(*) = 10.

The relcache must be loaded before the lock is taken (the two warm-up SELECTs). Otherwise the ALTER blocks earlier, while building the relcache entry, and never reaches the faulty code. A pg_cancel_backend() / statement_timeout, or any other ERROR raised in the same window, reproduces it too. The window is real in production: the lock wait just makes it deterministic.

Without the savepoint, i.e. a top-level ROLLBACK, the problem does not reproduce (see below for why).

Root cause

/* tablecmds.c, set_random_distribution_if_drop_distkey() */
rel->rd_cdbpolicy->nattrs = 0;                       /* (1) mutate the relcache entry in place */
...
policy = GpPolicyCopy(rel->rd_cdbpolicy);
GpPolicyReplace(RelationGetRelid(rel), policy);      /* (2) update gp_distribution_policy */
rel->rd_cdbpolicy = policy;

1. Step (1) writes an uncommitted, not-yet-CCI'd change straight into the relcache entry.
2. If an error is raised after (1) and before the next CommandCounterIncrement() (the one at the end of deleteOneObject()), the subtransaction aborts. AtEOSubXact_Inval(false) then replays only PriorCmdInvalidMsgs; the current command's messages are discarded. This is upstream PostgreSQL's design: it assumes that no cache yet reflects catalog changes the current command has not made visible. Step (1) breaks that assumption, so the stale entry (nattrs = 0) survives ROLLBACK TO, while the catalog correctly still says distkey = 1.
3. On the retry, GpPolicyIsPartitioned(rel->rd_cdbpolicy) sees nattrs == 0, and the function treats the table as already random and returns without touching the catalog. pg_attribute marks column 1 dropped, and COMMIT makes it permanent, leaving the catalog policy pointing at a dropped column.

The problem does not show up at top level only because of 1073dbdf59b ("Invalidate CatCache in AbortTransaction to clear reader gang's cache", GPDB #14048). That commit makes AbortTransaction() call AtEOXact_Inval(true) on the QD and writer QEs, which also processes the current command's messages and happens to hide the bug. There is no equivalent for subtransaction abort.

Also note that RemoveAttributeById() in heap.c (around line 2165) has a second copy of the same logic, with the same in-place mutation. Today it is dead code: the first copy has already set nattrs = 0, so GpPolicyIsHashPartitioned() is false by the time it runs.

Proposed fix

Follow the normal PostgreSQL pattern: change the catalog, and let the invalidation machinery update the relcache.

1. In set_random_distribution_if_drop_distkey(), build the new policy as a copy (GpPolicyCopy() in the current memory context, then set nattrs = 0 on the copy). Call GpPolicyReplace(), and do not modify rel->rd_cdbpolicy or assign to it. Updating gp_distribution_policy already registers a relcache invalidation for localoid (inval.c, tupleRelId == GpPolicyRelationId). The CommandCounterIncrement() at the end of deleteOneObject() processes it locally, and the rebuild reads the new random policy.
   • Error before that CCI: the relcache was never touched, so it is consistent with the rolled-back catalog.
   • Error after that CCI: the messages are in PriorCmdInvalidMsgs, the subtransaction abort replays them, and the entry is rebuilt from the rolled-back catalog.
2. Remove the duplicate block in RemoveAttributeById(). Once (1) is in place it would no longer be dead: before the CCI, rd_cdbpolicy still shows the old key, so it would call GpPolicyReplace() a second time on the same tuple in the same command (tuple already updated by self).
3. Add the isolation2 spec above as a regression test, checking both AO and heap tables.

Two related spots use a similar pattern and are worth a look, though the window is much narrower because they don't mutate in place and a CCI follows soon after. Both call GpPolicyReplace() and then assign a new pointer to rel->rd_cdbpolicy before any CCI:

• ATExecExpandTable() / leaf-partition expand
• the CTAS path of ALTER TABLE ... SET DISTRIBUTED BY, which sets a temporary random policy and later restores original_policy

These could be fixed the same way, separately.

Operating System

Rocky/CentOS 7 (x86_64), demo cluster with 3 segments. The behavior is not OS-specific.

Anything else

The same issue was first found in a downstream fork, using gdb to inject a cancel at the same point. The lock-timeout reproduction above needs only SQL.

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct