Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

PrebuiltBundleStore.Sweep does not unwind at silo teardown — the prebuilt-bundle retention walk is the leaked FileSystem leaf

Open
  • Is something wrong?
#7,049 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
azure, csharp
Domain
backend, infrastructure

Research direction

Read MeshWeaver.Hosting.PrebuiltBundleStore.Sweep and its pool invocation, then inspect the per-step walks in PrebuiltBundleStore.Plan; the issue says this source was not locatable in the available partitions. Compare the retention pass outcome and ledger for the named pod and timestamp, and check the sibling AssemblyCacheGenerations.Sweep incident for the same call-site pattern. Done means identifying whether the pooled walk observes cancellation or whether a step stalled, then documenting the evidence and a scoped fix.

Written by the indexing model from the issue text.

Description

bug sev:M

Observed

At silo teardown, IoPoolSiloTeardown reported that pooled I/O on the FileSystem pool did not finish within the 30 s drain budget and named the leaf that did not unwind: MeshWeaver.Hosting.PrebuiltBundleStore+<>c__DisplayClass7_0.<Sweep>b__0 — the Sweep lambda of PrebuiltBundleStore, the store the prebuilt-bundle retention pass walks (PrebuiltBundleRetentionHostedService, Prebuilt Bundle Retention). The silo therefore released over live work — the documented teardown contract violation (Controlled I/O Pooling — a blocking leaf that WALKS).

Where and when

One occurrence, one pod (memex-portal-deployment-64d94ddd78-ptc6k, memex-cloud, 2026-10-21T19:13:56Z), logged at shutdown — one per pod shutdown, not a recurring runtime failure. This is the same log site as #5719 (SealedBundleFloorCache.Observe, filed) and the #2480 precedent (ShippedPrebuiltBundles.<SeedBundles>b__8 — a bundle-share walk with the pool token discarded at the call site). An unfiled 2026-10-07 incident at this site names the sibling MeshWeaver.Graph.Configuration.AssemblyCacheGenerations+<>c__DisplayClass5_0.<Sweep>b__0 (Admin/_LogIncident/d4bf671508c99738; its triage wrote no draft) — two retention-sweep leaves at this site within two weeks, third sweep-shaped leaf overall.

Impact

A shutdown-time leak of one pooled leaf: no user-visible failure is evidenced. The documented escalation risk of a straggler running while the scope is disposed is the teardown use-after-unload SIGSEGV shape (#613's exit=139). Low frequency, real defect.

Not established

  • The cause. The log names the leaf precisely; what inside Sweep outlived the drain is not read — the type was not locatable in the mesh's partitions, so the call-site shape is inferred from the documented leaf contract, not from source.
  • Whether a retention pass was mid-walk at that instant: no PrebuiltBundleRetention: pass outcome line (#6991) or _retention/ledger.txt entry for this pod at that timestamp was found in the mesh.
  • Whether it recurs on other shutdowns, and whether other pods' teardowns on this deployment show the same leaf.
  • How the site evaded PooledLeafObservesItsTokenGuard — the same question #5719 left open (the guard is a lexical scan of the lambda form and cannot see method-group/delegate arguments, and this registration lives in the memex/ root).

Candidate causes and the reading that separates them

  • A — the walk discards the pool token at the call site (the #2480 / #5719 shape). The retention pass is a directory walk over <PreWarm:PrebuiltBundleRoot> — one identity directory per platform build, measured at 482 directories / 13 398 MiB on this fleet's /data share on 2026-09-10 — so a walk whose steps never check the token holds the silo's teardown join for as long as the share takes. Predicts: recurrence on every shutdown that lands mid-pass. Reading: the source of PrebuiltBundleStore.Sweep's pooled invocation in MeshWeaver.Hosting settles it directly, and the guard-evasion question above narrows where to look.
  • B — the token is observed but a single step stalled past the budget on the Azure Files share (the doc's stated carve-out: a single syscall that itself stalls cannot be interrupted from managed code, and the leaf named is then a finding about the mount, not the leaf). Predicts: rare, coincident with share latency or a full/slow share, no clean recurrence. Reading: the same source plus the pass-outcome and ledger lines for this pod at 19:13 separates it — a pass that started and never wrote its outcome line supports B as much as A.
  • The sibling evidence cuts on A: if AssemblyCacheGenerations.Sweep (2026-10-07) shares the same call-site pattern, two independent retention sweeps written to the same shape points at the pattern, not at a one-off mount stall.

Where to look

MeshWeaver.Hosting.PrebuiltBundleStore — the Sweep lambda (<>c__DisplayClass7_0.<Sweep>b__0), its pool invocation, and the per-step shape of the identity/marker/surface walks in PrebuiltBundleStore.Plan; registration at memex/Memex.Portal.Shared/MemexConfiguration.cs → AddPrebuiltBundleRetention. Site record Admin/_LogIncident/site-4806d28cbbe5e043 (routed to this repository; site issue #5719). Related docs: Prebuilt Bundle Retention, Controlled I/O Pooling.

Evidence
Fingerprint 1de4961048ddc28a
Category MeshWeaver.Connection.Orleans.IoPoolSiloTeardown
Severity Error
Namespace memex-cloud
Pods memex-portal-deployment-64d94ddd78-ptc6k
Occurrences 1
First seen 2026-10-21 19:13:56Z
Last seen 2026-10-21 19:13:56Z
Routing not determined — no configured route matches the category MeshWeaver.Connection.Orleans.IoPoolSiloTeardown. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-21 19:13:56Z memex-portal-deployment-64d94ddd78-ptc6k fail: MeshWeaver.Connection.Orleans.IoPoolSiloTeardown[0]
      IoPoolSiloTeardown: pooled I/O did not finish within 00:00:30 — the silo is releasing over live work. A leaf ignored its cancellation token; fix the leaf, do not widen the budget. Did NOT report: FileSystem=1 [MeshWeaver.Hosting.PrebuiltBundleStore+<>c__DisplayClass7_0.<Sweep>b__0].

This log site's previous ticket, https://github.com/Systemorph/MeshWeaver/issues/5719, was closed about a different fingerprint, and this fault fired after that close. It is left closed: an occurrence of this fingerprint is no evidence that the fault it was closed for came back. This issue now tracks the log site, and later fingerprints of it fold in here.

Opened automatically from Admin/_LogIncident/1de4961048ddc28a. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site 4806d28cbbe5e043: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

Dominant language
C#
Stars
12
Forks
5
Avg merge
6h 29m
Merged PRs (30d)
914

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Systemorph/MeshWeaver

All issues in Systemorph/MeshWeaver

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.