Local Kong keeps idle upstream connections longer than PostgREST, causing sporadic 502 on POST/PATCH

Open
#6,674 0 comments 0 reactions 1 assignee View on GitHub

@7ttp is already working on this.

Since Sep 19, 2026.

Assessment

This issue has not been assessed yet.

Description

🐛 Bug supabase/cli
Affected area

Local development (supabase start)

Describe the bug

Kong keeps idle upstream connections to PostgREST longer than PostgREST keeps them open. When Kong reuses a connection PostgREST has just closed, nginx retries idempotent methods but not POST/PATCH, so those fail with a sporadic 502:

{"message":"An invalid response was received from the upstream server"}

This shows up as flaky integration tests: a different test file fails on each run, and every failing file passes when run alone.

Measurements

On a stack started by CLI 2.98.0 (public.ecr.aws/supabase/kong:2.8.1, PostgREST v14.10, macOS + Docker):

What Result
Kong's upstream_keepalive_idle_timeout (/usr/local/kong/.kong_env) 60s (also upstream_keepalive_pool_size = 60, KONG_NGINX_WORKER_PROCESSES=1)
Time until PostgREST closes an idle connection (measured from inside the Kong container, 3 runs) 35.1s / 42.1s / 58.1s (Warp's idle timeout, coarse ~30s timer)
Kong error log over ~3 days 68 upstream errors: upstream prematurely closed connection ×47, recv() failed (104: Connection reset by peer) ×21 — GET 38 / HEAD 4 / DELETE 2 / PATCH 7 / POST 17
502s in the access log over the same window 24 — all of them POST (17) or PATCH (7). No GET/HEAD/DELETE became a 502
503 / 504 in the same window 0. PostgREST logged no PGRST003 (pool timeout) either
PostgREST log at the exact minutes of the 502s nothing — no reconnect, no schema cache reload, no DB disconnect

So the pool is not exhausted and PostgREST is not restarting; Kong is simply handing a request to a connection the upstream has already closed. Non-idempotent methods are not retried, which is why only POST/PATCH surface.

Expected behavior

Kong's idle timeout for the PostgREST upstream should be shorter than PostgREST's, so Kong drops the connection before the upstream does.

Workaround

Setting the value below PostgREST's minimum (30s) fixes it:

docker exec supabase_kong_<project> sh -c \
  "sed -i 's/^upstream_keepalive_idle_timeout = .*/upstream_keepalive_idle_timeout = 20/' /usr/local/kong/.kong_env && kong reload"
Why this needs a fix in the CLI

There is no way to change this from config.toml. The CLI embeds Kong's environment variables with fixed values, and PostgREST has no server-side idle timeout setting (postgrest --example only exposes server-host / server-port / server-cors-allowed-origins / server-timing-enabled / server-unix-socket). Patching .kong_env from outside works but silently stops working whenever the CLI changes how the container is built.

Could the CLI either ship a shorter upstream_keepalive_idle_timeout for the PostgREST upstream, or expose it in config.toml?

System information
  • Supabase CLI: 2.98.0
  • Kong image: public.ecr.aws/supabase/kong:2.8.1
  • PostgREST: v14.10
  • OS: macOS (Docker Desktop)
Dominant language
TypeScript
Stars
2.4k
Forks
523
Avg merge
20h 47m
Merged PRs (30d)
243

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from supabase/cli

All issues in supabase/cli

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.