[C#] Support BinaryData-backed data URI serialization

Đang mở
  • Có gì không ổn?
#11,649 1 bình luận 2 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
48/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Ít trao đổi
Công nghệ
csharp
Lĩnh vực
compilers, tooling

Hướng nghiên cứu

Bắt đầu bằng cách đọc cách emitter C# hiện đang xử lý các byte đã mã hóa và phần công việc liên quan đến Base64 trong #11623. Xác định cách biểu diễn TypeSpec được đề xuất dưới dạng data-uri phù hợp với các union URL-hoặc-dữ liệu và quá trình tuần tự hóa được tạo. Được xem là hoàn tất khi có quá trình tuần tự hóa lazy dựa trên BinaryData, việc phân tích cú pháp và tài liệu chính xác, các URL thông thường không thay đổi, và hỗ trợ các framework đích được liệt kê.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

emitter:client:csharp feature

Problem

The C# client emitter supports Base64 and Base64Url encodings for bytes, but it does not support data URIs of the form:

data:<media-type>;base64,<encoded-data>

This is distinct from the Base64 JSON value discussed in #11623. A data URI is serialized as a JSON string that contains a media type, the Base64 marker, and the encoded payload. Some service properties also accept either a normal URL or a data URI in the same wire value. Without generator support, libraries generally model these properties as string or url. A convenience API that accepts BinaryData must then create and retain the complete data URI before the client
operation starts.

For large images and files, this creates two large UTF-16 strings:

string base64 = Convert.ToBase64String(data.ToMemory().Span);
string dataUri = $"data:{mediaType};base64,{base64}";

An attempted C# customization in openai-dotnet retained BinaryData and created the data URI only when the generated model was serialized. The customization had to replace generated properties and JsonModelWriteCore methods across six models:

  • Chat image content
  • Chat file content
  • Responses input image content
  • Responses input file content
  • Image generation input masks
  • Computer screenshot output

This behavior belongs in generated serialization rather than hand-maintained model serializers.

Suggested change

Add data URI support for encoded bytes in the C# emitter. A possible TypeSpec representation is:

@encode("data-uri")
scalar DataUriBytes extends bytes;

For properties that accept either a URL or embedded data, the encoded bytes could participate in a union:

union ImageLocation {
  url,
  DataUriBytes,
}

The exact TypeSpec representation is open for discussion. The important C# behavior is:

  • Retain BinaryData and its media type without creating the data URI during model construction.
  • Serialize a standards-compliant data URI string when writing JSON.
  • Do not retain the temporary encoded string after serialization.
  • Materialize and cache a string only when a string-valued model property is explicitly read.
  • Preserve ordinary URL values without Base64 processing.
  • Parse deserialized data URIs when binary access is required.
  • Generate documentation that no copy is made and mutable backing memory must remain unaltered until the client operation completes.
Possible C# implementation shape 
internal sealed class DataUriValue
{
    private string? _value;

    public DataUriValue(BinaryData bytes, string mediaType)
    {
        Bytes = bytes;
        MediaType = mediaType;
    }

    public DataUriValue(string value)
    {
        _value = value;
    }

    public BinaryData? Bytes { get; }

    public string? MediaType { get; }

    public string GetValue(bool cache)
    {
        if (_value is not null)
        {
            return _value;
        }

        string value = CreateDataUri(Bytes!, MediaType!);
        if (cache)
        {
            _value = value;
        }

        return value;
    }
}

Generated model serialization could use the non-caching path:

writer.WritePropertyName("image_url"u8);
writer.WriteStringValue(_imageValue.GetValue(cache: false));

A public or customized string property could use the caching path:

public string ImageUri
    => _imageValue?.GetValue(cache: true);

On .NET 8 and later, data URI creation can write directly into one
final string:

private static string CreateDataUri(
    BinaryData data,
    string mediaType)
{
    ReadOnlyMemory<byte> memory = data.ToMemory();
    const string dataPrefix = "data:";
    const string base64Prefix = ";base64,";
    int base64Length = checked(((memory.Length + 2) / 3) * 4);
    int prefixLength = checked(
        dataPrefix.Length
        + mediaType.Length
        + base64Prefix.Length);

    return string.Create(
        checked(prefixLength + base64Length),
        (memory, mediaType, base64Length),
        static (destination, state) =>
        {
            int offset = 0;
            dataPrefix.AsSpan().CopyTo(destination);
            offset += dataPrefix.Length;
            state.mediaType.AsSpan().CopyTo(destination[offset..]);
            offset += state.mediaType.Length;
            base64Prefix.AsSpan().CopyTo(destination[offset..]);
            offset += base64Prefix.Length;

            if (!Convert.TryToBase64Chars(
                state.memory.Span,
                destination[offset..],
                out int charsWritten)
                || charsWritten != state.base64Length)
            {
                throw new InvalidOperationException(
                    "Base64 encoding did not produce the expected output.");
            }
        });
}

The implementation location and generated model shape are open for discussion. A shared runtime helper may be preferable to emitting the support type into every library.

Target framework compatibility

The lazy value representation only requires APIs available to netstandard2.0.

The single-string encoding path uses
Convert.TryToBase64Chars, which is not available for
netstandard2.0. The emitter can use conditional compilation:

  • .NET 8 and later use string.Create and Convert.TryToBase64Chars.
  • netstandard2.0 and .NET Framework use the existing array-aware Convert.ToBase64String(byte[], int, int) path.
  • Non-array-backed memory on older targets retains the existing ToArray() fallback.

The lazy value and fallback implementation were compiled with .NET SDK 10.0.302 for both netstandard2.0 and .NET Framework 4.6.2 with zero warnings. The .NET Framework 4.6.2 build was executed and produced the expected data URI from a sliced ReadOnlyMemory<byte>.

Compatible fallback 
private static string CreateDataUri(
    BinaryData data,
    string mediaType)
{
    ReadOnlyMemory<byte> memory = data.ToMemory();

#if NET8_0_OR_GREATER
    // Single-string implementation shown above.
#else
    string base64;
    if (MemoryMarshal.TryGetArray(
        memory,
        out ArraySegment<byte> segment)
        && segment.Array is not null)
    {
        base64 = Convert.ToBase64String(
            segment.Array,
            segment.Offset,
            segment.Count);
    }
    else
    {
        base64 = Convert.ToBase64String(memory.ToArray());
    }

    return $"data:{mediaType};base64,{base64}";
#endif
}

Benchmark

Deferring data URI creation removes payload-dependent allocation from model construction. When serialization is required, creating a single final string reduces end-to-end allocation while avoiding retention of that string by the model. Allocation is reduced by 40 percent for the complete construction and serialization path. Model construction itself becomes independent of payload size.

Results
Method Payload size Mean Allocated
Previous eager creation 1 MB 1.413 ms 5.09 MB
Previous eager creation and serialization 1 MB 2.622 ms 6.36 MB
Lazy model construction 1 MB 39.26 ns 248 B
Lazy construction and serialization 1 MB 1.206 ms 3.82 MB
Previous eager creation 10 MB 12.982 ms 50.86 MB
Previous eager creation and serialization 10 MB 23.065 ms 63.58 MB
Lazy model construction 10 MB 42.23 ns 248 B
Lazy construction and serialization 10 MB 17.708 ms 38.15 MB
Previous eager creation 100 MB 160.930 ms 508.63 MB
Previous eager creation and serialization 100 MB 237.883 ms 635.78 MB
Lazy model construction 100 MB 43.47 ns 248 B
Lazy construction and serialization 100 MB 174.924 ms 381.47 MB
Methodology

The benchmarks used BenchmarkDotNet 0.15.8 on .NET 8.0.29 with two warmup iterations and five measurement iterations. The host was Windows 11 on an Intel Xeon Platinum 8370C under Hyper-V. Payloads of 1, 10, and 100 million bytes were created once as BinaryData. The eager baseline created the Base64 string and final data URI during model construction. The lazy path retained BinaryData and created a non-cached data URI during ModelReaderWriter serialization.

Benchmark comparison code 
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using OpenAI.Chat;
using System.ClientModel.Primitives;
using System.Reflection;

BenchmarkRunner.Run<DataUriFactoryBenchmarks>();

[MemoryDiagnoser]
[SimpleJob(warmupCount: 2, iterationCount: 5)]
public class DataUriFactoryBenchmarks
{
    private BinaryData _data = null!;
    private FieldInfo _imagePartField = null!;
    private FieldInfo _imageValueField = null!;
    private FieldInfo _cachedValueField = null!;

    [Params(1_000_000, 10_000_000, 100_000_000)]
    public int PayloadSize { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _data = BinaryData.FromBytes(
            new byte[PayloadSize],
            "image/png");

        _imagePartField = typeof(ChatMessageContentPart)
            .GetField(
                "_imageUri",
                BindingFlags.Instance | BindingFlags.NonPublic)!;

        _imageValueField = _imagePartField.FieldType
            .GetField(
                "_imageValue",
                BindingFlags.Instance | BindingFlags.NonPublic)!;

        _cachedValueField = _imageValueField.FieldType
            .GetField(
                "_value",
                BindingFlags.Instance | BindingFlags.NonPublic)!;
    }

    [Benchmark]
    public string PreviousEagerCreation()
    {
        string base64 = Convert.ToBase64String(
            _data.ToMemory().Span);

        return $"data:image/png;base64,{base64}";
    }

    [Benchmark(Baseline = true)]
    public int PreviousEagerCreationAndSerialization()
    {
        string base64 = Convert.ToBase64String(
            _data.ToMemory().Span);
        string dataUri = $"data:image/png;base64,{base64}";

        ChatMessageContentPart part =
            ChatMessageContentPart.CreateImagePart(
                _data,
                "image/png");

        object imagePart = _imagePartField.GetValue(part)!;
        object imageValue =
            _imageValueField.GetValue(imagePart)!;
        _cachedValueField.SetValue(imageValue, dataUri);

        return ModelReaderWriter.Write(part).ToMemory().Length;
    }

    [Benchmark]
    public ChatMessageContentPart LazyFactoryCreation()
    {
        return ChatMessageContentPart.CreateImagePart(
            _data,
            "image/png");
    }

    [Benchmark]
    public int LazyFactoryAndSerialization()
    {
        ChatMessageContentPart part =
            ChatMessageContentPart.CreateImagePart(
                _data,
                "image/png");

        return ModelReaderWriter.Write(part).ToMemory().Length;
    }
}

The virtualization environment may affect absolute timing. Allocation results are the stronger evidence.

Expected behavior

  • TypeSpec can express bytes serialized as a data URI string.
  • The C# emitter retains BinaryData until serialization rather than eagerly creating a large string.
  • Ordinary URL values remain supported for URL-or-data-URI properties.
  • Generated serialization does not cache temporary data URI strings.
  • Explicit string property access may materialize and cache the value.
  • The data URI includes the correct media type and Base64 payload.
  • Generated code supports netstandard2.0, .NET Framework 4.6.2, and modern target frameworks.
  • Generated documentation explains the borrowed-buffer lifetime.
  • Existing wire representations remain unchanged.

Related context

  • #11623 covers standard Base64 and Base64Url JSON values. It does not cover data URI strings.
  • This was identified while investigating large binary payload memory pressure in openai-dotnet#1276.
Ngôn ngữ chính
Java
Star
5.9k
Fork
421
Merge trung bình
2 ngày 3 giờ
Pull request đã merge (30 ngày)
176

Chuẩn bị môi trường

Mở trong Codespaces

Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của microsoft/typespec

Tất cả issue của microsoft/typespec

Issue tương tự

Thêm issue về Java

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.