`supabase/postgres:17.6.1.105` (amd64) — backend SIGSEGV during a heavy pgTAP file, CI-CPU-specific
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- docker, github-actions, postgresql
- Domain
- ci-cd, databases, testing-qa
Research direction
Start by running the full self-contained pgTAP file against public.ecr.aws/supabase/postgres:17.6.1.105 on an AVX-512-capable amd64 GitHub Actions runner, with the stated PGOPTIONS settings. Collect the runner lscpu data and a backend core or backtrace, then determine whether the failure is tied to the image build or dispatch path. Done means a confirmed fault or a reproducible reduced case with a documented mitigation.
Written by the indexing model from the issue text.
Description
Summary
The amd64 public.ecr.aws/supabase/postgres:17.6.1.105 image deterministically SIGSEGVs a backend process (signal 11) while running one large single-transaction pgTAP test file — but only on GitHub Actions ubuntu-latest runners. The byte-identical image on a local amd64 workstation (Intel i7-8850H, AVX2) runs the same file cleanly every time, and our aarch64 production build is unaffected. This points to a CPU-microarchitecture-dependent code path in the amd64 17.6.1.105 build (the runner CPU is AVX-512-capable; our local CPU is not — AVX-512 is our leading hypothesis but is not core-dump-confirmed).
Environment
| Image / build | CPU | Result | |
|---|---|---|---|
GitHub Actions ubuntu-latest |
public.ecr.aws/supabase/postgres:17.6.1.105 (amd64) |
AVX-512-capable x86_64 (Xeon/EPYC-class) | SIGSEGV, 6/6 runs |
| Local workstation | same image (amd64) | Intel i7-8850H (AVX2, no AVX-512) | clean, every run |
| Supabase production | aarch64 build — PostgreSQL 17.6 on aarch64-unknown-linux-gnu |
ARM64 | clean |
- Local
version():PostgreSQL 17.6 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 15.2.0, 64-bit - Invocation:
docker exec -i supabase_db_<project> psql -U postgres -d postgres -v ON_ERROR_STOP=1 < <file>.sql(Supabase CLI local stack; the file runs inside oneBEGIN … ROLLBACK).
Observed behavior
A single pgTAP file (plan(363), ~4300 lines, one long transaction in one backend) crashes the backend mid-run:
LOG: server process (PID …) was terminated by signal 11: Segmentation fault
LOG: terminating any other active server connections
… database system is in recovery mode
The cluster then enters recovery and takes down the sibling test files running after it.
What it is NOT (ruled out with evidence)
- Not OOM — signal 11 (not 9);
dmesgis empty; adding swap did not help. - Not JIT —
jit=offset per-connection viaPGOPTIONS(verified active viaSHOW jit); still crashed. - Not the outer plan shape — forcing plans with GUCs (
enable_bitmapscan/hashagg/hashjoin=off,max_parallel_workers_per_gather=0) did not change the crash. - Not the SQL of the crashing statement — the crash lands on a statement (
SELECT * FROM a SECURITY DEFINER SETOF-uuid helper) invoked by a role whoseEXECUTEon that function is revoked, so it errors42501at the ACL check before the function body runs. The identical call by a privileged role earlier in the same file (and the same query shape ~17× before) runs clean. So the fault is not inside that function's execution — it is a delayed manifestation of state accumulated earlier in the backend.
The accumulation that appears to trigger it
The file, in one long-lived backend, builds up an unusually heavy load before the crash: hundreds of set_eq / EXCEPT-based pgTAP assertions (each materializing temp result sets), a BYPASSRLS role, and repeated trigger DDL / RLS-policy evaluation, all in one transaction. The corruption seems laid down by that accumulation and surfaces one statement later. On a fresh backend (e.g. splitting the file so the tail runs in its own psql/backend) the crash locus changes — consistent with private-backend-memory corruption rather than a specific query.
Reproduction
The full test file is self-contained pgTAP and reproduces 6/6 on an AVX-512-capable amd64 runner. We do not yet have a reduced repro (bisecting requires iterating on the AVX-512 hardware, which we only have via CI). We can share the full file privately, or work with you on a minimal case. To attempt repro:
supabase start(or runpublic.ecr.aws/supabase/postgres:17.6.1.105directly) on an AVX-512-capable amd64 host.- Apply a schema with RLS + a
SECURITY DEFINERset-returning helper, then run a long single-transaction pgTAP file that accumulates several hundredset_eq/EXCEPTassertions + aBYPASSRLSrole + trigger DDL. - Observe the backend SIGSEGV partway through.
A build compiled -O2 with CPU-dispatched SIMD (or a gcc 15.2.0 codegen issue on AVX-512) is the working hypothesis; a backtrace/core from the crashing backend on the runner would confirm the faulting routine.
Impact & current workaround
- We have quarantined the one file from CI and gated it locally on AVX2; all other tests pass. No production impact — production is aarch64 (a different build with no x86 SIMD path), and the helper executes correctly there.
- Asking here because it is your image build: a fix in the amd64 17.6.1.x build would let everyone re-enable such tests on standard CI runners.
Questions
- Is a SIGSEGV in the
17.6.1.105amd64 build on AVX-512 hardware a known issue? - Any recommended mitigation short of pinning an older/newer 17.x image (a build flag, a disabled dispatch path)?
- Would a full repro file + a runner
lscpube useful, or do you want a reduced case first?
- Dominant language
- Nix
- Stars
- 1.8k
- Forks
- 266
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 25
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from supabase/postgres
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
All issues in supabase/postgres
Similar issues
-
A cancelled tests run makes the coverage comment workflow fail and reports it as a red check on main Openarea: ci bug perceived difficulty: 3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Nitjsefnie-Harness-Commons/daedalus#921 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
crossplane/crossplane#7859 ·
-
Product: Terraform (AVM) Topic: Networking (HS) :globe_with_meridians:
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Azure/Azure-Landing-Zones#4282 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
bug carvel-triage
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
carvel-dev/kapp-controller#1861 ·