Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

OpenRouter prompt-cache policy OOMs while serializing an oversized tool message

Open
  • Is something wrong?
#6,112 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp
Domain
ai, backend

Research direction

Start at src/MeshWeaver.AI.OpenAI/OpenRouterPromptCachePolicy.cs, line 88, and inspect Apply alongside the OpenRouter prompt-caching feature notes. Then examine the cited thread cells to identify the tool result being serialized; the issue leaves open whether the policy or thread builder should bound it. Done means oversized tool results no longer cause this serialization OOM, with appropriate verification for the chosen fix.

Written by the indexing model from the issue text.

Description

bug sev:M

What is failing

When an OpenRouter-backed agent thread sends a chat round, the prompt-cache pipeline policy (OpenRouterPromptCachePolicy.Apply) forces the ChatCompletionOptions to be fully serialized to the request stream. On one thread (Hosting/Triage/_Thread/bug-systemorph-meshweaver-plugins-2403) that serialization ran out of memory while writing a ToolChatMessage's content string — the managed heap could not allocate the escaped copy of a single, enormous tool-result payload.

Probable cause

High confidence that the trigger is an oversized tool-call result (a tool message whose content is large enough that its JSON-escaped representation cannot be allocated on the pod's heap): the OOM lands in GC.AllocateNewArray called from Utf8JsonWriter.WriteStringEscapeValue via OpenAI.Chat.ChatMessageContent.WriteTo, i.e. the writer choked on one string, not on the sheer count of messages. Medium confidence on the defect itself: the caching policy (added with OpenRouter prompt caching, see Doc/WhatsNew/2026-08-18-openrouter-prompt-caching) re-serializes the whole conversation prefix each round, so an unbounded tool result is round-tripped through JSON every time — either the policy should cap/truncate what it serializes, or the thread builder should bound tool-result content before it enters the options. Whether this particular payload was a one-off dump or a runaway accumulation in that thread's history needs a look at the thread's cells.

Impact

One agent-thread round failed on one pod — the thread reported an execution error and the round did not reach the provider. No evidence of pod-wide memory pressure from this fingerprint (the same-day OOMs elsewhere are on other pods and other log sites). Not an outage, but any thread that ingests a large tool output will hit the same wall.

Where to look

Evidence
Fingerprint bd2e2f8c00ed87c4
Category MeshWeaver.AI.AgentChatClient
Severity Error
Exception System.OutOfMemoryException
Top frame OpenAI.Chat.ChatMessageContent.WriteTo(Utf8JsonWriter writer, ModelReaderWriterOptions options)
Namespace memex
Pods memex-portal-deployment-67c67b456d-s7nt9
Occurrences 1
First seen 2026-10-04 22:17:56Z
Last seen 2026-10-04 22:17:56Z
Routing not determined — no configured route matches the category MeshWeaver.AI.AgentChatClient. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject.
Recent log lines 
2026-10-04 22:17:56Z memex-portal-deployment-67c67b456d-s7nt9 fail: MeshWeaver.AI.AgentChatClient[0]
      [ThreadExec] ERROR: 22:17:56.197 threadPath=Hosting/Triage/_Thread/bug-systemorph-meshweaver-plugins-2403
      System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
         at System.GC.AllocateNewArray(IntPtr typeHandlePtr, Int32 length, GC_ALLOC_FLAGS flags, ObjectHandleOnStack ret)
         at System.GC.AllocateNewArray(IntPtr typeHandlePtr, Int32 length, GC_ALLOC_FLAGS flags, ObjectHandleOnStack ret)
         at System.GC.<AllocateUninitializedArray>g__AllocateNewArrayWorker|77_0[T](Int32 length, Boolean pinned)
         at System.Text.Json.Utf8JsonWriter.WriteStringEscapeValue(ReadOnlySpan`1 value, Int32 firstEscapeIndexVal)
         at OpenAI.Chat.ChatMessageContent.WriteTo(Utf8JsonWriter writer, ModelReaderWriterOptions options)
         at OpenAI.Chat.ToolChatMessage.WriteCore(Utf8JsonWriter writer, ModelReaderWriterOptions options)
         at OpenAI.Chat.ToolChatMessage.System.ClientModel.Primitives.IJsonModel<OpenAI.Chat.ToolChatMessage>.Write(Utf8JsonWriter writer, ModelReaderWriterOptions options)
         at OpenAI.ModelSerializationExtensions.WriteObjectValue[T](Utf8JsonWriter writer, T value, ModelReaderWriterOptions options)
         at OpenAI.Chat.ChatCompletionOptions.JsonModelWriteCore(Utf8JsonWriter writer, ModelReaderWriterOptions options)
         at OpenAI.Chat.ChatCompletionOptions.System.ClientModel.Primitives.IJsonModel<OpenAI.Chat.ChatCompletionOptions>.Write(Utf8JsonWriter writer, ModelReaderWriterOptions options)
         at System.ClientModel.Internal.ModelWriter`1.ExtractReader()
         at System.ClientModel.BinaryContent.ModelBinaryContent`1.get_SequenceReader()
         at System.ClientModel.BinaryContent.ModelBinaryContent`1.WriteTo(Stream stream, CancellationToken cancellation)
         at MeshWeaver.AI.OpenAI.OpenRouterPromptCachePolicy.Apply(PipelineMessage message) in /repo/src/MeshWeaver.AI.OpenAI/OpenRouterPromptCachePolicy.cs:line 88
        …[truncated]

Opened automatically from Admin/_LogIncident/bd2e2f8c00ed87c4. Recurrences are folded into this issue rather than opening new ones.
It also stands for the whole log site d12f6a93c8cbe683: other fingerprints of this site fold in here as comments rather than opening tickets of their own.

Dominant language
C#
Stars
12
Forks
5
Avg merge
3h 59m
Merged PRs (30d)
975

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Systemorph/MeshWeaver

All issues in Systemorph/MeshWeaver

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.