Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

PostgreSql mesh query provider stalls past the 15s Initial fan-in bound — Steward review rounds admitted without their concurrency bound

Closed
  • Is something wrong?
#6,104 7 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp, postgresql
Domain
backend, databases

Research direction

Read the Initial-emission path in PostgreSqlPartitionedMeshQuery and the fan-in merge that throws QueryProviderStalledException; check the IIoPool queue-wait metrics and MeshWeaver#1198 for evidence of waits exceeding 15 seconds. Then inspect the slot-claim error handling in PlatformBuildInboxWatcher and check whether Plugins#1798 is related. Done means the provider stall and fail-open admission are understood and addressed or ruled out; the issue leaves the cause and intended failure policy unresolved.

Written by the indexing model from the issue text.

Description

bug sev:M

What is failing

The Postgres mesh query provider (MeshWeaver.Hosting.PostgreSql.PostgreSqlPartitionedMeshQuery) does not emit an Initial within the query fan-in's 15-second bound, so the merged query has no snapshot and is reported unavailable (retryable). When this hits the Steward's review-slot claim, the slot for the round cannot be claimed and the round is admitted UNBOUNDED — i.e. without the concurrency bound the slot is supposed to enforce.

Probable cause (moderate confidence)

The provider appears starved of Postgres I/O capacity rather than erroring: the same provider-stall signature recurs across deployments over the past three weeks, and Postgres pool saturation is a known open issue (MeshWeaver#1198 — IIoPool queue-wait on the Postgres read/write pools; live instrumentation at rbuergi/Script/iopool-queue-wait). No stack frames beneath the exception are present in the captured samples, so the starvation link is inferred from the recurrence pattern, not read from a trace — treat as a lead, not a conclusion.

A second point needing a maintainer's ruling: on this availability failure the Steward degrades fail-open — it admits the round unbounded rather than failing the claim closed. If the bound is a safety/resource limit, skipping it on a transient query failure is its own defect.

Impact

This burst is a single event on one pod; the affected query is retryable by design, so no user-visible outage. However, the underlying provider stall is fleet-recurring, intermittently making mesh queries unavailable, and each stall at this site lets a review round run without its slot bound. If the provider stalls start bursting, a human should re-rule severity upward.

Where to look

  • The Initial-emission path of PostgreSqlPartitionedMeshQuery and the fan-in merge that throws QueryProviderStalledException (the 15s bound is the symptom site).
  • The IIoPool queue-wait metrics for the pg:Postgres / pg-read:Postgres pools (rbuergi/Script/iopool-queue-wait, MeshWeaver#1198) — check whether waits exceeded 15s at the time of the stall.
  • The Steward's slot-claim error handling in PlatformBuildInboxWatcher — decide whether fail-open unbounded admission is intended; a prior exception inside this watcher was tracked as Plugins#1798 (check if related/still open).

Possible duplicate of / related to MeshWeaver#1198 if the provider stall is confirmed to be pool starvation; filed separately because the fail-open unbounded admission at the watcher site is not covered there.

Evidence
Fingerprint 71040cc8c1afdf31
Category PlatformBuildInboxWatcher
Severity Error
Exception MeshWeaver.Mesh.QueryProviderStalledException
Namespace memex
Pods memex-portal-deployment-76dd69c7fc-xzsdl
Occurrences 1
First seen 2026-10-04 20:11:51Z
Last seen 2026-10-04 20:11:51Z
Routing not determined — no configured route matches the category PlatformBuildInboxWatcher. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-04 20:11:51Z memex-portal-deployment-76dd69c7fc-xzsdl fail: PlatformBuildInboxWatcher[0]
      [Steward] the review slot for Hosting/Triage/pull-request/systemorph-meshweaver-plugins-2867 at b68a766111a9f8f03ef994a4d48203b413b41076 could not be claimed on Hosting/Triage/Status — the round is admitted UNBOUNDED
      MeshWeaver.Mesh.QueryProviderStalledException: Query provider(s) [MeshWeaver.Hosting.PostgreSql.PostgreSqlPartitionedMeshQuery] did not emit an Initial within the query fan-in's 15s bound for query 'path:Hosting/Triage/Status' (user 'system-security'). The merged Initial gates on EVERY provider, so this query has NO snapshot to answer with — it is reported as unavailable (retryable) rather than left hanging with no error. This is an availability failure, never a permission verdict: a consumer deciding access must fail CLOSED and say it could not establish the answer. Fix the stalled provider; never bump the consumer's timeout.

Opened automatically from Admin/_LogIncident/71040cc8c1afdf31. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site 58c8829cb989c85c: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

Dominant language
C#
Stars
12
Forks
5
Avg merge
6h
Merged PRs (30d)
967

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Systemorph/MeshWeaver

All issues in Systemorph/MeshWeaver

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.