SqlTransaction.Dispose() throws NullReferenceException from SqlInternalTransaction.Rollback() when it races the background close of a broken connection

Abierto
#4,971 2 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 2 días

@MaceWindu ya está trabajando en esto.

Desde el 4/11/2026.

  • #4972 de @MaceWindu — abierto

Evaluación

Este issue todavía no se ha evaluado.

Descripción

Describe the bug

A command can fail with a fatal error while it is pending asynchronously inside a local transaction. Two examples are a network failure, and a command timeout whose attention the server never acknowledges (an unresponsive server). Disposing the transaction straight afterwards, in a using block or an ORM closing its data context, can then throw:

Exception message: Object reference not set to an instance of an object.
Stack trace:
   at Microsoft.Data.SqlClient.SqlInternalTransaction.Rollback()
   at Microsoft.Data.SqlClient.SqlInternalTransaction.Dispose(Boolean disposing)
   at Microsoft.Data.SqlClient.SqlInternalTransaction.Dispose()
   at Microsoft.Data.SqlClient.SqlTransaction.Dispose(Boolean disposing)

Or, when the race goes the other way:

Exception message: Invalid operation. The connection is closed.
Stack trace:
   at Microsoft.Data.SqlClient.SqlInternalTransaction.Rollback()
   at Microsoft.Data.SqlClient.SqlInternalTransaction.Dispose(Boolean disposing)
   ...

Either exception replaces the original SqlException (the timeout or the transport error) that the caller was handling.

We hit the NullReferenceException in production after query timeouts against a SQL Server that had stopped responding. The caller was linq2db's DataConnection.CloseAsync() → DbTransaction.DisposeAsync(). In that trace the frames between Rollback() and the caller are inlined.

Cause

On a fatal error the driver faults the pending operation first, and closes the connection afterwards on another thread:

  • Network failure: TdsParser.ThrowExceptionAndWarning(..., asyncClose: true) completes the task with the exception, then runs SqlConnection.Close() from Task.Factory.StartNew.
  • Unacknowledged timeout: the Task.Delay(AttentionTimeoutSeconds * 1000).ContinueWith(...) in TdsParserStateObject.OnTimeoutCore calls source.TrySetException(...), then _parser.Connection.BreakConnection(), which calls SqlConnection.Close().

So the caller's dispose runs concurrently with that close. SqlInternalTransaction.Dispose(bool) checks _innerConnection != null and calls Rollback(). Rollback() then reads _innerConnection again, first _innerConnection.IsLockedForBulkCopy (!_parser.MARSOn && ...), then ValidateConnectionForExecute, without the parser lock.

The close zombies the transaction (_innerConnection = null) and then disposes the internal connection (_parser = null). If that happens between the check and the dereference, the result is a NullReferenceException. If the connection is broken but the close has not run yet, ValidateConnectionForExecute throws ClosedConnectionError. Both exceptions are raised outside the try whose catch honours _disposing, so they escape Dispose().

The code is the same on main. #2879 and #1121 (NullReferenceException in GetServerTransactionLevel) look like the same family: _innerConnection cleared by a concurrent close while a transaction operation runs.

To reproduce

The program below runs a pending ExecuteScalarAsync inside BeginTransaction() through a TCP proxy. It breaks the connection, either with a reset or by freezing the proxy, then disposes the transaction immediately. The race window is a few instructions wide, so an EventListener on Microsoft.Data.SqlClient.EventSource sleeps on the disposing thread when the SqlInternalTransaction.Rollback | API scope event fires. That event is emitted after the null check in Dispose(bool) and before the dereference. No SqlClient code is modified. In production the same delay comes from thread-pool scheduling.

Results against SQL Server 2016 on localhost, .NET 10, Windows:

Version mode hold Outcome
7.0.0 reset 0 ms 9/10 InvalidOperationException: … connection is closed
7.0.0 reset 300 ms 7/10 NullReferenceException in Rollback()
7.1.1 reset 0 ms 4/10 InvalidOperationException: … connection is closed
7.1.1 reset 300 ms 2/10 NullReferenceException in Rollback()
7.0.0 freeze (timeout) 0 ms 2/8 InvalidOperationException: … connection is closed
7.0.0 freeze (timeout) 300 ms 2/8 NullReferenceException in Rollback()

The remaining runs are ones where the background close finished before Dispose() started, so the dispose skipped the rollback. Both UseCompatibilityAsyncBehaviour / UseCompatibilityProcessSni settings reproduce it.

Repro.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Data.SqlClient" Version="7.1.1" />
  </ItemGroup>
</Project>
Program.cs
using System.Collections.Concurrent;
using System.Diagnostics.Tracing;
using System.Net;
using System.Net.Sockets;
using Microsoft.Data.SqlClient;

// Repro: SqlTransaction.Dispose() throws NullReferenceException from SqlInternalTransaction.Rollback() (or
// InvalidOperationException "connection is closed") when it races the background close of a connection that
// broke under a pending async command.
//
// The EventListener below sleeps on the disposing thread when SqlClient writes the
// "SqlInternalTransaction.Rollback | API" trace event, which is emitted after Dispose's null check and before
// Rollback's dereference. That widens the race window; no SqlClient code is changed. holdMs=0 shows the race
// without the hold.
//
// Usage: dotnet run -- "<connection string>" [holdMs=300] [iterations=10] [mode=reset|freeze] [pauseMs=0] [compat=false]
//   mode reset:  TCP reset while the command is pending
//   mode freeze: the server stops responding; CommandTimeout (1 s) expires, the attention is never
//                acknowledged, and after the 5 s attention timeout SqlClient faults the task, then closes the connection
//   pauseMs:     random 0..pauseMs delay between the failure and Dispose
//   compat:      value of both UseCompatibilityAsyncBehaviour and UseCompatibilityProcessSni
string target = args[0];
int holdMs = args.Length > 1 ? int.Parse(args[1]) : 300;
int iterations = args.Length > 2 ? int.Parse(args[2]) : 10;
string mode = args.Length > 3 ? args[3] : "reset";
int pauseMs = args.Length > 4 ? int.Parse(args[4]) : 0;
bool compat = args.Length > 5 && bool.Parse(args[5]);
AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseCompatibilityAsyncBehaviour", compat);
AppContext.SetSwitch("Switch.Microsoft.Data.SqlClient.UseCompatibilityProcessSni", compat);
var random = new Random(1);

var targetBuilder = new SqlConnectionStringBuilder(target);
string[] hostPort = targetBuilder.DataSource.Split(',');

using var proxy = new ResettingProxy(hostPort[0], hostPort.Length > 1 ? int.Parse(hostPort[1]) : 1433);
proxy.Start();
var viaProxy = new SqlConnectionStringBuilder(target) { DataSource = $"127.0.0.1,{proxy.Port}", Pooling = false };

using var hold = new RollbackHold();
var outcomes = new ConcurrentDictionary<string, int>();
string? firstNre = null;

for (int i = 0; i < iterations; i++)
{
    var connection = new SqlConnection(viaProxy.ConnectionString);
    await connection.OpenAsync();
    var transaction = connection.BeginTransaction();

    var pending = Task.Run(async () =>
    {
        using var command = new SqlCommand("WAITFOR DELAY '00:00:30'; SELECT 1", connection, transaction) { CommandTimeout = 1 };
        await command.ExecuteScalarAsync();
    });
    await Task.Delay(200);
    if (mode == "freeze")
    {
        proxy.Frozen = true; // the server stops responding, the socket stays open
    }
    else
    {
        proxy.ResetAll(); // the network drops while the async command is pending
    }

    string operation;
    try { await pending; operation = "no error"; }
    catch (Exception ex) { operation = ex.GetType().Name + (ex is SqlException se ? $" {se.Number}" : ""); }

    if (pauseMs > 0)
    {
        await Task.Delay(random.Next(0, pauseMs));
    }

    // Dispose the transaction straight after the failure, as a using block or an ORM does.
    RollbackHold.HoldMs = holdMs;
    string dispose;
    try { transaction.Dispose(); dispose = "ok"; }
    catch (Exception ex)
    {
        dispose = $"{ex.GetType().Name}: {ex.Message}";
        if (ex is NullReferenceException) firstNre ??= ex.ToString();
    }
    finally { RollbackHold.HoldMs = 0; }

    connection.Dispose();
    proxy.ResetAll();
    proxy.Frozen = false;
    outcomes.AddOrUpdate($"command: {operation}; transaction.Dispose(): {dispose}", 1, (_, n) => n + 1);
}

var version = typeof(SqlConnection).Assembly
    .GetCustomAttributes(typeof(System.Reflection.AssemblyInformationalVersionAttribute), false)
    .OfType<System.Reflection.AssemblyInformationalVersionAttribute>().First().InformationalVersion;
Console.WriteLine($"Microsoft.Data.SqlClient {version}; holdMs={holdMs}");
foreach (var (outcome, count) in outcomes) Console.WriteLine($"{count,4} x {outcome}");
if (firstNre != null) Console.WriteLine(firstNre);

internal sealed class RollbackHold : EventListener
{
    public static volatile int HoldMs;

    protected override void OnEventSourceCreated(EventSource source)
    {
        if (source.Name == "Microsoft.Data.SqlClient.EventSource")
        {
            EnableEvents(source, EventLevel.Verbose, EventKeywords.All);
        }
    }

    protected override void OnEventWritten(EventWrittenEventArgs e)
    {
        if (HoldMs > 0 && e.Payload is { Count: > 0 } p && p[0] is string s
            && s.StartsWith("SqlInternalTransaction.Rollback | API", StringComparison.Ordinal))
        {
            Thread.Sleep(HoldMs);
        }
    }
}

// Forwards TCP to the server; ResetAll() aborts every forwarded connection with RST.
internal sealed class ResettingProxy(string host, int port) : IDisposable
{
    private readonly TcpListener _listener = new(IPAddress.Loopback, 0);
    private readonly ConcurrentDictionary<int, (TcpClient Client, TcpClient Server)> _live = new();
    private int _ids;
    public volatile bool Frozen;

    public int Port => ((IPEndPoint)_listener.LocalEndpoint).Port;

    public void Start()
    {
        _listener.Start();
        _ = Task.Run(async () =>
        {
            while (true)
            {
                TcpClient client;
                try { client = await _listener.AcceptTcpClientAsync(); }
                catch (Exception) { return; }
                _ = Task.Run(() => Forward(client));
            }
        });
    }

    private async Task Forward(TcpClient client)
    {
        int id = Interlocked.Increment(ref _ids);
        var server = new TcpClient();
        try
        {
            await server.ConnectAsync(host, port);
            _live[id] = (client, server);
            await Task.WhenAny(Pump(client.GetStream(), server.GetStream()), Pump(server.GetStream(), client.GetStream()));
        }
        catch (Exception) { }
        finally
        {
            _live.TryRemove(id, out _);
            client.Close();
            server.Close();
        }
    }

    private async Task Pump(NetworkStream from, NetworkStream to)
    {
        var buffer = new byte[16384];
        try
        {
            int n;
            while ((n = await from.ReadAsync(buffer)) > 0)
            {
                while (Frozen) await Task.Delay(10);
                await to.WriteAsync(buffer.AsMemory(0, n));
            }
        }
        catch (Exception) { }
    }

    public void ResetAll()
    {
        foreach (var id in _live.Keys)
        {
            if (_live.TryRemove(id, out var pair))
            {
                try { pair.Client.Client.LingerState = new LingerOption(true, 0); pair.Client.Close(); } catch (Exception) { }
                try { pair.Server.Close(); } catch (Exception) { }
            }
        }
    }

    public void Dispose() => _listener.Stop();
}
Expected behavior

SqlTransaction.Dispose() does not throw because the connection broke. Nothing can be rolled back on a broken connection, the server ends the transaction with the session, and _disposing exists so that dispose-time rollback failures are not thrown.

Further technical details

Microsoft.Data.SqlClient version: 7.0.0 (production), 7.1.1 and main (reproduced)
.NET target: .NET 10
SQL Server version: SQL Server 2019 (production), SQL Server 2016 (repro)
Operating system: Windows Server (production), Windows 10 (repro)

Additional context

I have a fix with deterministic regression tests and will open a PR for it.

Lenguaje dominante
C#
Estrellas
1k
Forks
352
Merge medio
4 d 8 h
PR fusionados (30 d)
100

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dotnet/SqlClient

Todos los issues de dotnet/SqlClient

Issues similares

Más issues de C#

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.