Hosted hub creation for sync child hubs fails with OutOfMemoryException under portal memory pressure

クローズ
  • 何か問題がありますか？
#6,166 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
44/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
csharp, kubernetes
領域
backend, infrastructure

調査の方向性

Start with src/MeshWeaver.Messaging.Hub/HostedHubsCollection.cs at CreateHub and MessageHubConfiguration.cs at Build, then inspect pod memory limits and GC/container metrics for memex-portal around the recorded failure. Compare with the linked prior OOM incidents to assess whether this is recurring memory pressure; done means the failure cause is established and an appropriate remediation or monitoring action is identified.

索引モデルが issue の本文から書いたものです。

説明

bug sev:M

What is failing

Creating a hosted child hub for a sync/... address on the portal fails: HostedHubsCollection.CreateHub dies with a System.OutOfMemoryException while MessageHubConfiguration.Build spins up the hub's DI scope. The exception is thrown inside Autofac — StartableManager.StartStartableComponents resolving services and populating the registration tracker — so the hub never starts and the sync work it was created for never runs.

Probable cause

Most likely pod-level memory exhaustion, moderate-to-low confidence in the specific trigger and high confidence in the failure site. The stack shows no MeshWeaver allocation loop — the OOM surfaces inside Autofac's resolve pipeline while it initializes service registrations for the new hub's scope, which is exactly where a process already at its memory limit fails next. An OOM thrown from DI-container bookkeeping (not from buffering or serializing a payload) is a symptom of the pod having no headroom at that moment, rather than an unbounded allocation in CreateHub itself. Notably, the memex-portal deployment has a history of OOMs from memory-hungry operations — see the message-batch serialization OOM (Admin/_LogIncident/4a50cc9363466ae2) and the plugin-publish buffering OOM (Admin/_LogIncident/7bb4ddf387689fad, filed as Systemorph/MeshWeaver#5501) — so cumulative memory pressure on this deployment is plausible. This incident is a distinct log site from those and is not a duplicate ticket, but the three together suggest the portal pods run close to their limit.

Impact

Judged from the numbers: a single occurrence on one pod — an intermittent, instance-scoped failure, not an outage. When it hits, the sync child hub is not created, so the synchronization that requested it stalls until retried. The larger risk is what it signals: a portal pod at its memory ceiling, where the next allocation failure could hit a primary path instead of hub creation. If this recurs, treat it as a pod-stability / memory-provisioning problem rather than a one-off.

Where to look

  • src/MeshWeaver.Messaging.Hub/HostedHubsCollection.cs, CreateHub (~line 540) and MessageHubConfiguration.cs, Build (~line 470) — the creation path that fails; check whether hub construction can be made resilient (retry, or fail fast with a clearer error) when the pod is under memory pressure.
  • Pod memory requests/limits and GC/container metrics for the memex-portal deployment around the failure window — determine whether the pod was near its limit and whether the earlier OOM incidents (linked above) are the same underlying pressure.
  • Background on the hub topology these sync child hubs belong to: Message-Based Communication.
Evidence
Fingerprint 324501c685a9f636
Category MeshWeaver.Messaging.HostedHubsCollection
Severity Error
Exception System.OutOfMemoryException
Top frame Autofac.Extensions.DependencyInjection.AutofacRegistration.<>c__DisplayClass2_0.<AddFromKeyedServiceParameterMiddleware>b__0(Object sender, IResolvePipelineBuilder pipeline)
Namespace memex
Pods memex-portal-deployment-6cfcf78898-gkkdv
Occurrences 1
First seen 2026-10-05 16:41:55Z
Last seen 2026-10-05 16:41:55Z
Routing not determined — no configured route matches the category MeshWeaver.Messaging.HostedHubsCollection. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-05 16:41:55Z memex-portal-deployment-6cfcf78898-gkkdv fail: MeshWeaver.Messaging.HostedHubsCollection[0]
      Failed to create hosted hub for address sync/U6rl__80XUS6ctcFdmR-5A
      System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
         at Autofac.Extensions.DependencyInjection.AutofacRegistration.<>c__DisplayClass2_0.<AddFromKeyedServiceParameterMiddleware>b__0(Object sender, IResolvePipelineBuilder pipeline)
         at Autofac.Core.Registration.DefaultRegisteredServicesTracker.PopulateServiceInfo(Service service, ServiceRegistrationInfo info, Boolean isScopeIsolatedService)
         at Autofac.Core.Registration.DefaultRegisteredServicesTracker.GetInitializedServiceInfo(Service service)
         at Autofac.Core.Registration.DefaultRegisteredServicesTracker.ServiceRegistrationsFor(Service service)
         at Autofac.Builder.StartableManager.StartStartableComponents(IDictionary`2 properties, IComponentContext componentContext)
         at MeshWeaver.Messaging.MessageHubConfiguration.Build[TAddress](IServiceProvider serviceProvider, TAddress address) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Messaging.Hub/MessageHubConfiguration.cs:line 470
         at MeshWeaver.Messaging.HostedHubsCollection.CreateHub(Address address, Func`2 config) in /home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Messaging.Hub/HostedHubsCollection.cs:line 540
Out of memory.

Opened automatically from Admin/_LogIncident/324501c685a9f636. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site a76c620f59f02e6d: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

主要言語
C#
スター
12
フォーク
5
平均マージ
5時間 7分
マージ済み PR（30日）
981

環境構築

  • Dockerfile・Docker Compose ファイルなし
  • プルリクエストのテンプレートあり
  • コントリビューションガイドなし

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

Systemorph/MeshWeaver のほかの issue

Systemorph/MeshWeaver の issue をすべて見る

似ている issue

C# の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。