Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Add segmented buffering and sequence reads to `ArrayPoolBufferWriter<T>`

Open
  • Is something wrong?
#1,214 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
csharp
Domain
backend, performance

Research direction

Start by inspecting the existing ArrayPoolBufferWriter implementation, especially GetSpan(), GetMemory(), WrittenMemory, WrittenSpan, DangerousGetArray(), Clear(), and Dispose(). Compare the proposed segmented behavior with the feature/segmented-arraypoolbufferwriter branch. Done means adding GetReadOnlySequence(), retaining completed pooled arrays, consolidating contiguous views on demand, and returning all arrays during cleanup.

Written by the indexing model from the issue text.

Description

feature request :mailbox_with_mail:
Overview

ArrayPoolBufferWriter<T> currently copies all written data into a new pooled array whenever it grows. With multiple writes and an unknown final size, those repeated copies consume memory bandwidth. Retaining completed arrays as segments would avoid copying earlier writes on each growth and let callers read the result as a ReadOnlySequence<T> without flattening it. Callers that need a contiguous buffer could still use the existing APIs, which would consolidate on demand.

API breakdown
namespace CommunityToolkit.HighPerformance.Buffers;

public sealed class ArrayPoolBufferWriter<T> : IBuffer<T>, IMemoryOwner<T>
{
    public ReadOnlySequence<T> GetReadOnlySequence();
}

Existing public signatures remain unchanged. GetSpan() and GetMemory() write to the active segment; WrittenMemory, WrittenSpan, and DangerousGetArray() provide contiguous data through on-demand consolidation. Clear() and Dispose() return retained arrays to the pool. A returned sequence is valid only until the writer is modified, consolidated, cleared, or disposed.

Usage example
using ArrayPoolBufferWriter<byte> writer = new();

foreach (ReadOnlyMemory<byte> chunk in chunks)
{
    chunk.Span.CopyTo(writer.GetSpan(chunk.Length));
    writer.Advance(chunk.Length);
}

// Consume the segments while the writer still owns their pooled arrays.
foreach (ReadOnlyMemory<byte> segment in writer.GetReadOnlySequence())
{
    destination.Write(segment.Span);
}
Breaking change?

I'm not sure

Alternatives

Keep the current contiguous writer and pay the copy cost on growth, or use Microsoft.IO.RecyclableMemoryStream where its MemoryStream-like API and pooling features are needed.

Additional context

This may be slower for some workloads. Segmentation adds buffer rents and metadata, while requesting a contiguous view still requires a copy. Small payloads and code that frequently accesses WrittenMemory or WrittenSpan should be benchmarked alongside append-heavy workloads.

Combined with other CommunityToolkit.HighPerformance memory and stream helpers, this might be a good alternative to Microsoft.IO.RecyclableMemoryStream for some append-and-read workloads. It would not be a drop-in replacement for that library’s seekable stream behavior, configurable pooling, or diagnostics. A proposed implementation is in feature/segmented-arraypoolbufferwriter.

Help us help you

Yes, I'd like to be assigned to work on this item

Dominant language
C#
Stars
3.8k
Forks
400
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from CommunityToolkit/dotnet

All issues in CommunityToolkit/dotnet

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.