[C#] Support BinaryData-backed data URI serialization

Aperta
  • Qualcosa non va?
#11,649 1 commento 2 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
csharp
Ambito
compilers, tooling

Direzione di ricerca

Inizia leggendo la gestione esistente dei byte codificati nell’emettitore C# e il lavoro correlato su Base64 in #11623. Determina come la rappresentazione TypeSpec proposta come data-uri si adatti alle unioni URL-o-dati e alla serializzazione generata. Il lavoro è completato quando sono disponibili la serializzazione lazy basata su BinaryData, il parsing e la documentazione corretti, gli URL ordinari invariati e il supporto per i framework di destinazione elencati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

emitter:client:csharp feature

Problem

The C# client emitter supports Base64 and Base64Url encodings for bytes, but it does not support data URIs of the form:

data:<media-type>;base64,<encoded-data>

This is distinct from the Base64 JSON value discussed in #11623. A data URI is serialized as a JSON string that contains a media type, the Base64 marker, and the encoded payload. Some service properties also accept either a normal URL or a data URI in the same wire value. Without generator support, libraries generally model these properties as string or url. A convenience API that accepts BinaryData must then create and retain the complete data URI before the client
operation starts.

For large images and files, this creates two large UTF-16 strings:

string base64 = Convert.ToBase64String(data.ToMemory().Span);
string dataUri = $"data:{mediaType};base64,{base64}";

An attempted C# customization in openai-dotnet retained BinaryData and created the data URI only when the generated model was serialized. The customization had to replace generated properties and JsonModelWriteCore methods across six models:

  • Chat image content
  • Chat file content
  • Responses input image content
  • Responses input file content
  • Image generation input masks
  • Computer screenshot output

This behavior belongs in generated serialization rather than hand-maintained model serializers.

Suggested change

Add data URI support for encoded bytes in the C# emitter. A possible TypeSpec representation is:

@encode("data-uri")
scalar DataUriBytes extends bytes;

For properties that accept either a URL or embedded data, the encoded bytes could participate in a union:

union ImageLocation {
  url,
  DataUriBytes,
}

The exact TypeSpec representation is open for discussion. The important C# behavior is:

  • Retain BinaryData and its media type without creating the data URI during model construction.
  • Serialize a standards-compliant data URI string when writing JSON.
  • Do not retain the temporary encoded string after serialization.
  • Materialize and cache a string only when a string-valued model property is explicitly read.
  • Preserve ordinary URL values without Base64 processing.
  • Parse deserialized data URIs when binary access is required.
  • Generate documentation that no copy is made and mutable backing memory must remain unaltered until the client operation completes.
Possible C# implementation shape 
internal sealed class DataUriValue
{
    private string? _value;

    public DataUriValue(BinaryData bytes, string mediaType)
    {
        Bytes = bytes;
        MediaType = mediaType;
    }

    public DataUriValue(string value)
    {
        _value = value;
    }

    public BinaryData? Bytes { get; }

    public string? MediaType { get; }

    public string GetValue(bool cache)
    {
        if (_value is not null)
        {
            return _value;
        }

        string value = CreateDataUri(Bytes!, MediaType!);
        if (cache)
        {
            _value = value;
        }

        return value;
    }
}

Generated model serialization could use the non-caching path:

writer.WritePropertyName("image_url"u8);
writer.WriteStringValue(_imageValue.GetValue(cache: false));

A public or customized string property could use the caching path:

public string ImageUri
    => _imageValue?.GetValue(cache: true);

On .NET 8 and later, data URI creation can write directly into one
final string:

private static string CreateDataUri(
    BinaryData data,
    string mediaType)
{
    ReadOnlyMemory<byte> memory = data.ToMemory();
    const string dataPrefix = "data:";
    const string base64Prefix = ";base64,";
    int base64Length = checked(((memory.Length + 2) / 3) * 4);
    int prefixLength = checked(
        dataPrefix.Length
        + mediaType.Length
        + base64Prefix.Length);

    return string.Create(
        checked(prefixLength + base64Length),
        (memory, mediaType, base64Length),
        static (destination, state) =>
        {
            int offset = 0;
            dataPrefix.AsSpan().CopyTo(destination);
            offset += dataPrefix.Length;
            state.mediaType.AsSpan().CopyTo(destination[offset..]);
            offset += state.mediaType.Length;
            base64Prefix.AsSpan().CopyTo(destination[offset..]);
            offset += base64Prefix.Length;

            if (!Convert.TryToBase64Chars(
                state.memory.Span,
                destination[offset..],
                out int charsWritten)
                || charsWritten != state.base64Length)
            {
                throw new InvalidOperationException(
                    "Base64 encoding did not produce the expected output.");
            }
        });
}

The implementation location and generated model shape are open for discussion. A shared runtime helper may be preferable to emitting the support type into every library.

Target framework compatibility

The lazy value representation only requires APIs available to netstandard2.0.

The single-string encoding path uses
Convert.TryToBase64Chars, which is not available for
netstandard2.0. The emitter can use conditional compilation:

  • .NET 8 and later use string.Create and Convert.TryToBase64Chars.
  • netstandard2.0 and .NET Framework use the existing array-aware Convert.ToBase64String(byte[], int, int) path.
  • Non-array-backed memory on older targets retains the existing ToArray() fallback.

The lazy value and fallback implementation were compiled with .NET SDK 10.0.302 for both netstandard2.0 and .NET Framework 4.6.2 with zero warnings. The .NET Framework 4.6.2 build was executed and produced the expected data URI from a sliced ReadOnlyMemory<byte>.

Compatible fallback 
private static string CreateDataUri(
    BinaryData data,
    string mediaType)
{
    ReadOnlyMemory<byte> memory = data.ToMemory();

#if NET8_0_OR_GREATER
    // Single-string implementation shown above.
#else
    string base64;
    if (MemoryMarshal.TryGetArray(
        memory,
        out ArraySegment<byte> segment)
        && segment.Array is not null)
    {
        base64 = Convert.ToBase64String(
            segment.Array,
            segment.Offset,
            segment.Count);
    }
    else
    {
        base64 = Convert.ToBase64String(memory.ToArray());
    }

    return $"data:{mediaType};base64,{base64}";
#endif
}

Benchmark

Deferring data URI creation removes payload-dependent allocation from model construction. When serialization is required, creating a single final string reduces end-to-end allocation while avoiding retention of that string by the model. Allocation is reduced by 40 percent for the complete construction and serialization path. Model construction itself becomes independent of payload size.

Results
Method Payload size Mean Allocated
Previous eager creation 1 MB 1.413 ms 5.09 MB
Previous eager creation and serialization 1 MB 2.622 ms 6.36 MB
Lazy model construction 1 MB 39.26 ns 248 B
Lazy construction and serialization 1 MB 1.206 ms 3.82 MB
Previous eager creation 10 MB 12.982 ms 50.86 MB
Previous eager creation and serialization 10 MB 23.065 ms 63.58 MB
Lazy model construction 10 MB 42.23 ns 248 B
Lazy construction and serialization 10 MB 17.708 ms 38.15 MB
Previous eager creation 100 MB 160.930 ms 508.63 MB
Previous eager creation and serialization 100 MB 237.883 ms 635.78 MB
Lazy model construction 100 MB 43.47 ns 248 B
Lazy construction and serialization 100 MB 174.924 ms 381.47 MB
Methodology

The benchmarks used BenchmarkDotNet 0.15.8 on .NET 8.0.29 with two warmup iterations and five measurement iterations. The host was Windows 11 on an Intel Xeon Platinum 8370C under Hyper-V. Payloads of 1, 10, and 100 million bytes were created once as BinaryData. The eager baseline created the Base64 string and final data URI during model construction. The lazy path retained BinaryData and created a non-cached data URI during ModelReaderWriter serialization.

Benchmark comparison code 
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using OpenAI.Chat;
using System.ClientModel.Primitives;
using System.Reflection;

BenchmarkRunner.Run<DataUriFactoryBenchmarks>();

[MemoryDiagnoser]
[SimpleJob(warmupCount: 2, iterationCount: 5)]
public class DataUriFactoryBenchmarks
{
    private BinaryData _data = null!;
    private FieldInfo _imagePartField = null!;
    private FieldInfo _imageValueField = null!;
    private FieldInfo _cachedValueField = null!;

    [Params(1_000_000, 10_000_000, 100_000_000)]
    public int PayloadSize { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _data = BinaryData.FromBytes(
            new byte[PayloadSize],
            "image/png");

        _imagePartField = typeof(ChatMessageContentPart)
            .GetField(
                "_imageUri",
                BindingFlags.Instance | BindingFlags.NonPublic)!;

        _imageValueField = _imagePartField.FieldType
            .GetField(
                "_imageValue",
                BindingFlags.Instance | BindingFlags.NonPublic)!;

        _cachedValueField = _imageValueField.FieldType
            .GetField(
                "_value",
                BindingFlags.Instance | BindingFlags.NonPublic)!;
    }

    [Benchmark]
    public string PreviousEagerCreation()
    {
        string base64 = Convert.ToBase64String(
            _data.ToMemory().Span);

        return $"data:image/png;base64,{base64}";
    }

    [Benchmark(Baseline = true)]
    public int PreviousEagerCreationAndSerialization()
    {
        string base64 = Convert.ToBase64String(
            _data.ToMemory().Span);
        string dataUri = $"data:image/png;base64,{base64}";

        ChatMessageContentPart part =
            ChatMessageContentPart.CreateImagePart(
                _data,
                "image/png");

        object imagePart = _imagePartField.GetValue(part)!;
        object imageValue =
            _imageValueField.GetValue(imagePart)!;
        _cachedValueField.SetValue(imageValue, dataUri);

        return ModelReaderWriter.Write(part).ToMemory().Length;
    }

    [Benchmark]
    public ChatMessageContentPart LazyFactoryCreation()
    {
        return ChatMessageContentPart.CreateImagePart(
            _data,
            "image/png");
    }

    [Benchmark]
    public int LazyFactoryAndSerialization()
    {
        ChatMessageContentPart part =
            ChatMessageContentPart.CreateImagePart(
                _data,
                "image/png");

        return ModelReaderWriter.Write(part).ToMemory().Length;
    }
}

The virtualization environment may affect absolute timing. Allocation results are the stronger evidence.

Expected behavior

  • TypeSpec can express bytes serialized as a data URI string.
  • The C# emitter retains BinaryData until serialization rather than eagerly creating a large string.
  • Ordinary URL values remain supported for URL-or-data-URI properties.
  • Generated serialization does not cache temporary data URI strings.
  • Explicit string property access may materialize and cache the value.
  • The data URI includes the correct media type and Base64 payload.
  • Generated code supports netstandard2.0, .NET Framework 4.6.2, and modern target frameworks.
  • Generated documentation explains the borrowed-buffer lifetime.
  • Existing wire representations remain unchanged.

Related context

  • #11623 covers standard Base64 and Base64Url JSON values. It does not cover data URI strings.
  • This was identified while investigating large binary payload memory pressure in openai-dotnet#1276.
Lingua principale
Java
Stelle
5.9k
Fork
421
Merge medio
2g 3h
PR unite (30g)
176

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di microsoft/typespec

Tutte le issue di microsoft/typespec

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.