@databricks/lakebase: pool drops maxLifetimeSeconds and other pg options, so connections never recycle and Unity Catalog claims access fails after 4 hours
Maintainers usually reply within 3 days
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 20/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- node.js, typescript
Research direction
Read packages/lakebase/src/pool-config.ts and inspect getLakebasePgConfig() to see which PoolConfig options reach pg.Pool. Check the pool tests for coverage of forwarded options and the default connection lifetime. Done means caller options are preserved, connections recycle at the intended default, and the warning behavior covers unsafe lifetime settings; linked PR #719 is already open.
Written by the indexing model from the issue text.
Description
Summary
createLakebasePool() passes only 11 options to pg.Pool and silently drops the rest, including maxLifetimeSeconds. Pool connections are therefore never recycled.
Lakebase checks the OAuth token only at login, so most queries keep working on old connections. Access granted through Unity Catalog claims in the token doesn't: it stops 4 hours after the token is minted, while the connection stays open. A busy pool keeps its connections past that point. From then on, every query that relies on that access fails with 42501 permission denied, until the app restarts.
The same bug drops application_name, keepAlive, query_timeout and options. LakebasePoolConfig extends pg's PoolConfig, so all of these type-check and are then ignored.
Affected: every published @databricks/lakebase version, up to and including 0.7.0, and AppKit's createLakebasePool, which wraps it.
Reproduction
The dropped options show without a database. pg.Pool doesn't connect until first use:
import { createLakebasePool } from "@databricks/lakebase";
const pool = createLakebasePool({
host: "localhost",
database: "test",
user: "test",
password: "test-only",
sslMode: "disable",
maxLifetimeSeconds: 900,
application_name: "my-app",
query_timeout: 15_000,
});
console.log(pool.options.maxLifetimeSeconds); // 0 (pg-pool's "never recycle"), not 900
console.log(pool.options.application_name); // undefined
console.log(pool.options.query_timeout); // undefined
await pool.end();
The access lapse, on a live Lakebase Autoscaling endpoint:
- Use a service principal with an OAuth Postgres role and no native
GRANTs. Give itSELECTon a synced table through Unity Catalog only. - Create a pool whose password callback returns a token minted with
claims: [{ permission_set: "READ_ONLY", resources: [{ table_name: "<catalog.schema.table>" }] }]. - Keep a few connections busy, reading the table every 30 seconds, for more than 4 hours.
- At about 240 minutes after the token was minted, every read fails with
42501 permission denied for schema …. Liveness queries on the same connections keep succeeding, and the pool never recovers.
In our run, connections were 4.3 hours old by the end, and the lapse came at 240.5 minutes on every connection. With 1-hour tokens, reads kept working for 15 minutes past the token's expiry, and with 5-minute tokens for 29 minutes. So the lapse follows the token's age (4 hours from mint), not its expiry.
Expected
- Options a caller passes reach
pg.Pool. - Connections are recycled by default, early enough that no connection outlives its token's Unity Catalog claims.
Root cause
getLakebasePgConfig() in packages/lakebase/src/pool-config.ts builds the pool config from a fixed list:
return {
host, port, user, database, password, ssl, max,
idleTimeoutMillis, connectionTimeoutMillis,
statement_timeout, idle_in_transaction_session_timeout,
};
There's no maxLifetimeSeconds, so pg-pool uses its default of 0, which means connections are never closed for age.
Proposed fix
- Pass
maxLifetimeSeconds,application_name,keepAlive,query_timeoutandoptionsthrough. - Default
maxLifetimeSecondsto 30 minutes. - Log a warning when it's
0or long enough for a connection to outlive its token's claims. A token can be up to an hour old at login, so with a 30-minute margin the safe maximum is 2.5 hours.
A PR follows.
Notes
createLakebasePooldoesn't request claims itself, so this hits apps that mint claims tokens (generateDatabaseCredentialacceptsclaims) and pass them as the pool password. Access through native Postgres grants isn't affected.- pg-pool recycles only idle connections. A client that's checked out and held for hours isn't closed until it's released, so the fix can't protect it.
- Dominant language
- TypeScript
- Stars
- 119
- Forks
- 36
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 37
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/appkit
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
databricks/appkit#729 ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
databricks/appkit#699 ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 Half a day Newbie friendliness 42/100
databricks/appkit#656 · 1 comment ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 28/100
databricks/appkit#648 · 2 comments ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
databricks/appkit#603 ·
Maintainers usually reply within 3 days
All issues in databricks/appkit
Similar issues
-
Difficulty 1/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
jupyterlite/jupyterlite#2111 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
karlhorky/electron-app-patcher#241 ·
Maintainers usually reply within 1 day
-
Tab bar scrolls past its last tab in Firefox until no tabs are visiblePossibly taken @daviwil claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
eclipse-theia/theia#18367 ·
Maintainers usually reply within 2 days
-
Server-side render fails with "requestAnimationFrame is not defined" when the room has a video nodePossibly taken @jesusmpc claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
InditexTech/weavejs#1294 ·
Maintainers usually reply within 1 day