Flaky test: a_run_whose_retained_definition_does_not_restore_is_held_alone reopens the deployment while the audit sink is still locked

Open Beginner friendly
#2,706 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, postgresql, rust
Domain
databases, testing

Research direction

Start with the failing test a_run_whose_retained_definition_does_not_restore_is_held_alone in crates/registry-coordinator/tests/deployment_postgres.rs, especially the teardown and deployment::open_ctl call near line 5665. Run the Coordinator PostgreSQL runtime test and check how the test can wait for the previous audit sink lock to be released or retry open_ctl on sink-locked for a bounded time. Done means the test handles that teardown race without relaxing the product refusal.

Written by the indexing model from the issue text.

Description

What happened

The Coordinator PostgreSQL runtime job failed in a merge group for an unrelated change (#2705, run 44574520469, 19 Nov 2026). The same job passed on that PR's own run.

test a_run_whose_retained_definition_does_not_restore_is_held_alone ... FAILED
panicked at crates/registry-coordinator/tests/deployment_postgres.rs:5665:67:
called `Result::unwrap()` on an `Err` value: CoordinatorError { code: "deployment.refused",
  message: "the deployment contract was refused", ...,
  suggested_action: Some("restore writable durable audit custody (sink-locked)"), ... }
test result: FAILED. 16 passed; 1 failed; 3 ignored

Likely cause

Just before line 5665 the test aborts the server task (server.abort(); let _ = server.await;), drops the first migration store and calls deployment::open_ctl(&runtime_v2, true). The refusal says the durable audit sink was still locked. That suggests the lock held by the previous store or the aborted server is sometimes released after the reopen begins. This is a teardown ordering race in the test, not a product defect that we have observed outside it.

Suggested fix

Make the reopen wait until the previous holder has released the audit sink lock, for example by waiting for the lock release explicitly or retrying open_ctl on sink-locked for a bounded time in the test only. Do not relax the product's refusal.

Impact

Merge-queue entries that select the Coordinator PostgreSQL suite can be dropped. Rerunning the entry has passed so far.

Dominant language
Rust
Stars
2
Forks
0
Avg merge
9h 48m
Merged PRs (30d)
151

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from registrystack/registry-stack

All issues in registrystack/registry-stack

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.