Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Fleet coordinator schedule faults with ObjectDisposedException when the ledger stream's MeshNodeStreamCache is disposed mid-read

Closed
  • Is something wrong?
#6,222 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp
Domain
backend

Research direction

Start with FleetCoordinatorRuntime.cs, especially LedgerWakes and the self-healing wrapper described in the issue, then inspect the MeshNodeStreamCache dispose path in MeshWeaver mesh services. The issue points to a framework-level race but does not identify which teardown triggers it. Done means a live GetMeshNodeStream reader no longer leaks ObjectDisposedException on cache disposal; also check the related consumer in #6078.

Written by the indexing model from the issue text.

Description

bug sev:L

What is failing

The fleet coordinator's schedule — armed on the always-on Hosting/PlatformBuilds hub by FleetCoordinatorRuntime.AddFleetCoordinator — faults when the backing MeshNodeStreamCache of its long-lived read of the Hosting/Triage/Status ledger stream is disposed mid-subscription. The ObjectDisposedException tears down the merged schedule (round loop, startup, ledger wakes, heartbeat); the self-healing wrapper catches it and re-arms a minute later, so the coordinator is deaf to event wakes (red main, new bug, feedback, stuck) for up to that minute.

Probable cause

Medium-high confidence on the mechanism, lower on the trigger. The coordinator holds node-stream subscriptions for the whole life of the hub: LedgerWakes samples hub.GetMeshNodeStream(CoordinatorSnapshot.StatusPath) — the Hosting/Triage/Status ledger — continuously. When the mesh-node cache backing that node is torn down underneath the reader (a node-hub recycle/quiesce or hub re-activation), the read throws instead of re-acquiring. This is the same dispose race as #6078 (the review-slot watcher, same exception, same object name, different consumer) and the same family as #5958; what this incident cannot establish is which teardown disposed the cache at the moment of the fault — the window shows Hosting/Coordination and Hosting/PlatformBuildInbox type recycles minutes before, but no single event lines up exactly. The residual defect is the framework's: a long-lived GetMeshNodeStream reader should survive an owning hub's teardown transparently or fail quietly — leaking ObjectDisposedException into consumers is the bug, and the coordinator's self-healing only masks it at this one consumer.

Impact

Judged from the numbers: a single occurrence on a single pod of the control (memex) deployment, self-healed by the designed one-minute restart. The design absorbs it — the heartbeat (default 30 min) and the next wake cover the gap, rounds are coalesced not lost, and no round was in flight (the round loop's own Catch logs a different message). Internal control plane only; no user-facing path, no data loss. Cost: at most a one-minute delay in event-driven prioritisation.

Not a duplicate, but closely related. #6078 is the same defect at the PullRequestSweep consumer (different log category, different fingerprint site) and remains open; if the fix lands at the MeshNodeStreamCache level, both consumers heal together and this can be closed as such.

Where to look

  • FleetCoordinatorRuntime.cs — AddFleetCoordinator: the TriageIntake.SelfHealing wrapper that logs this fault and re-arms after a minute, and hub.RegisterForDisposal(signal); LedgerWakes: the long-lived GetMeshNodeStream on Hosting/Triage/Status that throws.
  • The framework's MeshNodeStreamCache (MeshWeaver mesh services): its dispose path and whether a live reader can observe it — whether GetMeshNodeStream should re-acquire after the owning hub's teardown rather than throw. Same seam as #6078.
Evidence
Fingerprint 19ba7381a30e3924
Category FleetCoordinator
Severity Error
Exception System.ObjectDisposedException
Namespace memex
Pods memex-portal-deployment-54ff68d87d-f5hp9
Occurrences 1
First seen 2026-10-07 03:10:04Z
Last seen 2026-10-07 03:10:04Z
Routing not determined — no configured route matches the category FleetCoordinator. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-07 03:10:04Z memex-portal-deployment-54ff68d87d-f5hp9 fail: FleetCoordinator[0]
      [Coordinator] the schedule faulted on Hosting/PlatformBuilds — restarting in a minute
      System.ObjectDisposedException: The mesh-node cache was disposed; the read of 'Hosting/Triage/Status' ended with it.
      Object name: 'MeshNodeStreamCache'.

Opened automatically from Admin/_LogIncident/19ba7381a30e3924. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site fd9bb2e2c965b0a4: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

Dominant language
C#
Stars
12
Forks
5
Avg merge
3h 59m
Merged PRs (30d)
975

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Systemorph/MeshWeaver

All issues in Systemorph/MeshWeaver

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.