Orleans grain directory serves stale silo address after restart, briefly rejecting message-hub deliveries
Abierto
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- csharp
- Área
- distributed-systems
Línea de trabajo
Start by checking the memex-cloud silo 10.244.9.90 events around 2026-09-22T14:41Z, as the issue identifies this as the key way to confirm or refute the stale-directory hypothesis. Then inspect the IMessageHubGrain delivery path in MeshWeaver.Connection.Orleans and the Orleans lookup and placement stack named in the report. Done means establishing whether directory convergence or a silo restart explains the failures and identifying a supported fix; the issue does not specify a fix or tests.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
bug
sev:L
What is failing
The Orleans client on the portal pod (memex-portal-deployment-5c444645f8-8pdzb, memex-cloud) failed to deliver IMessageHubGrain.DeliverMessage requests to message-hub grain targets (e.g. {path}, {path}, {path}). Message placement failed because the client could not open a connection to the destination silo S10.244.9.90:11111:149090968; the messages were rejected with OrleansMessageRejectionException wrapping ConnectionFailedException.
Probable cause
Stale silo address in the grain directory / membership view (medium confidence). The destination silo's generation in the failing address (149090968) is older than the portal client's own generation (149091044) — the client was trying to reach a superseded silo activation whose address was still returned by LocalGrainDirectory.LookupAsync (stack runs through DhtGrainLocator.Lookup → PlacementService.PlacementWorker). All 28 failures landed within an ~8.6 s window and then stopped, consistent with directory convergence catching up after a silo restart rather than a persistent network partition. Not certain: if silo 10.244.9.90 was genuinely down at that moment, this is the transient expected during a rolling restart — check whether that silo restarted around 2026-09-22 14:41 UTC.
Impact
Minimal: 28 rejected messages over ~8.6 s on a single pod, no recurrence. Message delivery retries after the connect backoff (sub-second), so the user-visible effect is likely nil.
Where to look
MeshWeaver.Connection.Orleans— theIMessageHubGraindelivery path and how the portal client resolves message-hub grain placements.- Orleans runtime internals in the stack:
Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync,Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync,Orleans.Runtime.Placement.PlacementService. - Check memex-cloud silo
10.244.9.90{path} events around 2026-09-22T14:41Z to confirm or refute the stale-directory hypothesis.
Evidence
| Fingerprint | 7f51fc88b6154267 |
| Category | Orleans.Messaging |
| Severity | Error |
| Exception | Orleans.Runtime.OrleansMessageRejectionException |
Namespace memex |
6 occurrence(s), 2026-10-30 19:03:44Z → 2026-10-30 19:03:48Z; pods memex-portal-deployment-86f575cbd7-nkscr, memex-portal-deployment-86f575cbd7-642d5 |
| Pods (namespace not recorded) | memex-portal-deployment-5c444645f8-8pdzb, memex-portal-deployment-89868db67-xg22d, memex-portal-deployment-56fbdcd48f-4dpcn, memex-portal-deployment-7d7f4d84f9-mvzb5, memex-portal-deployment-745745db7c-b2gjn, memex-portal-deployment-8d85b9854-s66rp, memex-portal-deployment-8d85b9854-gtckf, memex-portal-deployment-57b7d56fdd-5wqxq, memex-portal-deployment-6f8489747-fchl7, memex-portal-deployment-6f8489747-49hwv, memex-portal-deployment-748f7f4577-d92xf, memex-portal-deployment-748f7f4577-pjrjp, memex-portal-deployment-748f7f4577-tp7b6, memex-portal-deployment-76cddc5847-x9fff, memex-portal-deployment-76cddc5847-f4gk9, memex-portal-deployment-5779f6fc79-g7ckx, memex-portal-deployment-5779f6fc79-kf2hr, memex-portal-deployment-d677f4c4d-cskdv, memex-portal-deployment-d677f4c4d-5x827, memex-portal-deployment-7868c55d6-z27mm, memex-portal-deployment-7868c55d6-jbp2d, memex-portal-deployment-677bd84df8-4s2ww, memex-portal-deployment-7868c55d6-hfnt8, memex-portal-deployment-85f94799c4-lw5sv, memex-portal-deployment-85f94799c4-5pjlr, memex-portal-deployment-5fd6d747d-xbct6, memex-portal-deployment-78986f8447-m2gsv, memex-portal-deployment-78986f8447-2sgfl, memex-portal-deployment-78fbdddf6c-drjx8, memex-portal-deployment-57c8f675f9-z4bnj, memex-portal-deployment-7bcd666f5-bzc8n, memex-portal-deployment-7bcd666f5-vpv79 |
| Occurrences | 2090 |
| First seen | 2026-09-22 14:41:30Z |
| Last seen | 2026-10-30 19:03:48Z |
| Routing | not determined — no configured route matches the category Orleans.Messaging. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject. |
🚨 This incident covers 2 distinct message shapes, so its own Last seen is the newest of ALL of them. Read the row for the shape this issue is about — a shape whose last sighting predates a fix is not recurring, whatever the counter says.
| Shape | Occurrences | First seen | Last seen |
|---|---|---|---|
7f51fc88b6154267 — Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S1{n}.{n}:{n}:{n}, will retry after {n}ms |
6 | 2026-10-30 19:03:44Z | 2026-10-30 19:03:48Z |
445e1378ed560f40 — Failed to address message Request [S1{n}.{n}:{n}:{n} {path}:{n}@{n}]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messagi… |
2084 | 2026-09-22 14:41:30Z | 2026-10-23 22:41:34Z |
Recent log lines
2026-10-23 22:41:34Z memex-portal-deployment-7bcd666f5-vpv79 fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.6.71:11111:151799210 {path}:11111@151799210]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #40FAFBF56FF0289F
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.3.239:11111:151799091, will retry after 55.7015ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in {path}:li…[truncated]
2026-10-23 22:41:34Z memex-portal-deployment-7bcd666f5-vpv79 fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.6.71:11111:151799210 {path}:11111@151799210]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #40FAFBF56FF028BE
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.3.239:11111:151799091, will retry after 4.1972ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
at Orleans.R…[truncated]
2026-10-23 22:41:34Z memex-portal-deployment-7bcd666f5-bzc8n fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.13.103:11111:151799325 {path}:11111@151799325]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #5BFEE84DEFC3449E
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.3.239:11111:151799091, will retry after 336.1283ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 5…[truncated]
2026-10-23 22:41:34Z memex-portal-deployment-7bcd666f5-bzc8n fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.13.103:11111:151799325 {path}:11111@151799325]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #5BFEE84DEFC34490
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.3.239:11111:151799091, will retry after 267.7709ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in {path}…[truncated]
2026-10-23 22:41:34Z memex-portal-deployment-7bcd666f5-bzc8n fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.13.103:11111:151799325 {path}:11111@151799325]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #5BFEE84DEFC344EC
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.3.239:11111:151799091, will retry after 239.8539ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
2026-10-30 19:03:45Z memex-portal-deployment-86f575cbd7-nkscr fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.12.156:11111:152367314 {path}:11111@152367314]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #7A256BEBBDEFDB63
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.19.237:11111:152367582, will retry after 540.9723ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
2026-10-30 19:03:46Z memex-portal-deployment-86f575cbd7-nkscr fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.12.156:11111:152367314 {path}:11111@152367314]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #7A256BEBBDEFDB76
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.19.237:11111:152367582, will retry after 649.8368ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
2026-10-30 19:03:47Z memex-portal-deployment-86f575cbd7-nkscr fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.12.156:11111:152367314 {path}:11111@152367314]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #7A256BEBBDEFDB7F
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.19.237:11111:152367582, will retry after 301.9734ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
2026-10-30 19:03:47Z memex-portal-deployment-86f575cbd7-nkscr fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.12.156:11111:152367314 {path}:11111@152367314]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #7A256BEBBDEFDB85
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.19.237:11111:152367582, will retry after 742.1559ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
2026-10-30 19:03:48Z memex-portal-deployment-86f575cbd7-642d5 fail: Orleans.Messaging[100071]
Failed to address message Request [S10.244.18.120:11111:152367030 {path}:11111@152367030]->[ {path}] MeshWeaver.Connection.Orleans.IMessageHubGrain.DeliverMessage(MeshWeaver.Messaging.IMessageDelivery) #55B61060B70E2D95
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.244.19.237:11111:152367582, will retry after 96.6695ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 142
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|32_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 245
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 239
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
--- End of stack trace from previous location ---
at Orleans.Runtime.GrainDirectory.LocalGrainDirectory.LookupAsync(GrainId grainId, Int32 hopCount) in /_/src/Orleans.Runtime/GrainDirectory/LocalGrainDirectory.cs:line 902
at Orleans.Runtime.GrainDirectory.DhtGrainLocator.Lookup(GrainId grainId) in /_/src/Orleans.Runtime/GrainDirectory/DhtGrainLocator.cs:line 35
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.ExecutePlacementAsync(PlacementContext context, CancellationToken cancellationToken) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 602
at Orleans.Runtime.Placement.PlacementService.PlacementWorker.<ExecutePlacementOutcomeAsync>g__AwaitOutcome|21_0(ValueTask`1 pending) in /_/src/Orleans.Runtime/Placement/PlacementService.cs:line 576
…[truncated]
This log site's previous ticket, https://github.com/Systemorph/MeshWeaver/issues/5336, was closed about a different fingerprint, and this fault fired after that close. It is left closed: an occurrence of this fingerprint is no evidence that the fault it was closed for came back. This issue now tracks the log site, and later fingerprints of it fold in here.
Re-addressed by the current identity function: this incident inherited 445e1378ed560f40 (Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.
Opened automatically from Admin/_LogIncident/7f51fc88b6154267. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site 08481dbd6614b2ac: other fingerprints of this site fold in here as comments rather than opening tickets of their own.
- Lenguaje dominante
- C#
- Estrellas
- 12
- Forks
- 5
- Merge medio
- 5 h 28 min
- PR fusionados (30 d)
- 976
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Sin guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de Systemorph/MeshWeaver
-
Key exchange follow-ups: registry hook 500 ambiguity and an exchange health line (split from #7484)Abiertosev:M
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
Systemorph/MeshWeaver#7721 ·
Los mantenedores suelen responder en 1 día
-
bug sev:M
Dificultad 4/5 3-5 días Aptitud para principiantes 25/100
Systemorph/MeshWeaver#7720 ·
Los mantenedores suelen responder en 1 día
-
bug sev:M
Dificultad 4/5 3-5 días Aptitud para principiantes 25/100
Systemorph/MeshWeaver#7719 ·
Los mantenedores suelen responder en 1 día
-
bug sev:H
Dificultad 4/5 3-5 días Aptitud para principiantes 35/100
Systemorph/MeshWeaver#7702 ·
Los mantenedores suelen responder en 1 día
-
bug sev:M
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
Systemorph/MeshWeaver#7690 · 1 comentario ·
Los mantenedores suelen responder en 1 día
Todos los issues de Systemorph/MeshWeaver
Issues similares
-
请让 FPS 叠加层的字号大小可单独设置Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
Los mantenedores suelen responder en 1 día
-
امکان انتخاب متن پیام هرزنامهAbierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 66/100
MahdiyarGHD/GheychiApp#39 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
-
bug effort:S P3
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
nightscout/nocturne#3136 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100