Regression in 11.0: replacing an owned instance or re-adding an entity with the same key sets unchanged OnAdd properties to null

Start in ModificationCommand.ColumnValuePropagator.RecordValue and TryPropagate, identified in the issue’s explanation and stack trace; compare the behavior introduced around #37751 with the working 11.0.0-preview.2 version. Reproduce with the supplied SQLite example, then add or update a regression test for replacing an owned instance or deleting and re-adding an entity with the same key. Done means unchanged ValueGenerated.OnAdd properties retain their values after SaveChanges, and the existing update behavior remains covered.

Bug description

After upgrading from 10.0.15 to 11.0.0, SaveChanges started throwing this in a few of our write paths:

System.InvalidOperationException: The value for property 'Address.Floor' cannot be set to null because its type is 'int' which is not a nullable type.

It shows up when a deleted entry and an added entry map to the same row in one SaveChanges:

1. an owned reference (table splitting) is replaced with a new instance, or
2. an entity with a client-assigned key is removed and a new instance with the same key is added.

On top of that, the property has to be ValueGenerated.OnAdd (we see it with HasDefaultValue, HasDefaultValueSql and plain ValueGeneratedOnAdd()), and the new instance has to carry the same value as the row being replaced. If the value changes, it works. Properties that aren't OnAdd are not affected.

Nullable and string properties don't throw. Instead, the property on the tracked instance is null after SaveChanges. The column is left out of the UPDATE, so the row keeps the right value, and the entity in memory no longer matches the database. That one is easy to miss.

Results are the same on SQLite and on MySQL 8.4 with Microting.EntityFrameworkCore.MySql 11.0.0-preview.1, and 10.0.15 works with both, so it doesn't look provider specific. Going through the preview packages, 11.0.0-preview.2 works and 11.0.0-preview.3 fails. It still fails on 11.0.0.

It looks related to #37751. As far as I can tell, in ColumnValuePropagator.RecordValue the Deleted entry sets _write = true and _currentValue = null. The Added entry then hits the early break added there, because its value equals _originalValue, so _currentValue stays null. TryPropagate then treats the Added entry as stale and calls SetStoreGeneratedValue(entry, property, null). The early break seems to assume _originalValue/_currentValue were recorded by a Modified entry. When they come from a Deleted entry, there is no new value to propagate.

This broke several write paths for us in production and we rolled back to 10 for now, so it would be great to see it in an 11.0.x patch. In the meantime, .ValueGeneratedNever() on the affected properties works for us: the column default stays and EF always sends the value.

Your code

#:package [email protected]
#:property PublishAot=false

using Microsoft.EntityFrameworkCore;

using (var db = new AppDb())
{
    db.Database.EnsureDeleted();
    db.Database.EnsureCreated();
    db.Add(new Customer { Id = 1, Address = new Address { City = "Paris", Floor = 3, Note = "back door" } });
    db.Add(new OrderLine { OrderId = 1, ProductId = 10, Quantity = 5 });
    db.SaveChanges();
}

Run("Replace owned instance, Floor unchanged", db =>
{
    var customer = db.Customers.Single();
    customer.Address = new Address { City = "Lyon", Floor = 3, Note = "back door" };
    db.SaveChanges();
    Console.WriteLine($"  after save: Floor={customer.Address.Floor}, Note={customer.Address.Note ?? "<null>"}");
});

Run("Delete + add OrderLine with same key, Quantity unchanged", db =>
{
    db.Remove(db.OrderLines.Single());
    var line = new OrderLine { OrderId = 1, ProductId = 10, Quantity = 5 };
    db.Add(line);
    db.SaveChanges();
    Console.WriteLine($"  after save: Quantity={line.Quantity}");
});

Run("Replace owned instance, Floor changed, Note unchanged", db =>
{
    var customer = db.Customers.Single();
    customer.Address = new Address { City = "Nice", Floor = 4, Note = "back door" };
    db.SaveChanges();
    Console.WriteLine($"  after save: Floor={customer.Address.Floor}, Note={customer.Address.Note ?? "<null>"}");
});

using (var db = new AppDb())
{
    var customer = db.Customers.AsNoTracking().Single();
    Console.WriteLine($"in database: Floor={customer.Address.Floor}, Note={customer.Address.Note ?? "<null>"}");
}

static void Run(string name, Action<AppDb> action)
{
    Console.WriteLine(name);
    try
    {
        using var db = new AppDb();
        action(db);
    }
    catch (Exception e)
    {
        Console.WriteLine($"  {e.GetType().Name}: {e.Message}");
    }
}

public class Customer
{
    public int Id { get; set; }
    public Address Address { get; set; } = null!;
}

public class Address
{
    public string City { get; set; } = "";
    public int Floor { get; set; }
    public string? Note { get; set; }
}

public class OrderLine
{
    public int OrderId { get; set; }
    public int ProductId { get; set; }
    public int Quantity { get; set; }
}

public class AppDb : DbContext
{
    public DbSet<Customer> Customers => Set<Customer>();
    public DbSet<OrderLine> OrderLines => Set<OrderLine>();

    protected override void OnConfiguring(DbContextOptionsBuilder options)
        => options.UseSqlite("Data Source=repro.db");

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Customer>().OwnsOne(c => c.Address, a =>
        {
            a.Property(x => x.Floor).HasDefaultValue(1);
            a.Property(x => x.Note).HasDefaultValue("none");
        });

        modelBuilder.Entity<OrderLine>(b =>
        {
            b.HasKey(x => new { x.OrderId, x.ProductId });
            b.Property(x => x.Quantity).HasDefaultValue(1);
        });
    }
}

Output with 11.0.0:

Replace owned instance, Floor unchanged
  InvalidOperationException: The value for property 'Address.Floor' cannot be set to null because its type is 'int' which is not a nullable type.
Delete + add OrderLine with same key, Quantity unchanged
  InvalidOperationException: The value for property 'OrderLine.Quantity' cannot be set to null because its type is 'int' which is not a nullable type.
Replace owned instance, Floor changed, Note unchanged
  after save: Floor=4, Note=<null>
in database: Floor=4, Note=back door

Output with 10.0.15:

Replace owned instance, Floor unchanged
  after save: Floor=3, Note=back door
Delete + add OrderLine with same key, Quantity unchanged
  after save: Quantity=5
Replace owned instance, Floor changed, Note unchanged
  after save: Floor=4, Note=back door
in database: Floor=4, Note=back door

Stack traces

System.InvalidOperationException: The value for property 'Address.Floor' cannot be set to null because its type is 'int' which is not a nullable type.
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SidecarValues.SetValue(IProperty property, Object value, Int32 index)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetProperty(IPropertyBase propertyBase, Object value, Boolean isMaterialization, Boolean setModified, Boolean isCascadeDelete, CurrentValueType valueType)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetStoreGeneratedValue(IProperty property, Object value, Boolean setModified)
   at Microsoft.EntityFrameworkCore.Update.ColumnModification.SetStoreGeneratedValue(IUpdateEntry entry, IProperty property, Object value)
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.ColumnValuePropagator.TryPropagate(IColumnMappingBase mapping, IUpdateEntry entry)
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<GenerateColumnModifications>g__HandleColumn|40_3(IUpdateEntry entry, IColumnMappingBase columnMapping, Boolean nonMainEntry, Boolean& optionalDependentWithAllNull, <>c__DisplayClass40_0&)
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<GenerateColumnModifications>g__HandleNonJson|40_2(IUpdateEntry entry, ITypeBase structuralType, ITableMapping tableMapping, Boolean nonMainEntry, Boolean& optionalDependentWithAllNull, <>c__DisplayClass40_0&)
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.GenerateColumnModifications()
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<>c.<get_ColumnModifications>b__33_0(ModificationCommand command)
   at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func`2 valueFactory)
   at Microsoft.EntityFrameworkCore.Update.ModificationCommand.get_ColumnModifications()
   at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.CreateCommandBatches(IEnumerable`1 commandSet, Boolean moreCommandSets, Boolean assertColumnModification, ParameterNameGenerator parameterNameGenerator)+MoveNext()
   at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.BatchCommands(IList`1 entries, IUpdateAdapter updateAdapter)+MoveNext()
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.<SaveChanges>b__115_0(DbContext _, ValueTuple`2 t)
   at Microsoft.EntityFrameworkCore.Storage.NonRetryingExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges()
   at Program.<>c.<<Main>$>b__0_0(AppDb db) in repro.cs:line 19
   at Program.<<Main>$>g__Run|0_3(String name, Action`1 action) in repro.cs:line 52

Verbose output

Third case: the UPDATE doesn't touch Address_Note, but customer.Address.Note is null after SaveChanges.

UPDATE "Customers" SET "Address_City" = @p0, "Address_Floor" = @p1
WHERE "Id" = @p2
RETURNING 1;

EF Core version

11.0.0 (also 11.0.0-preview.3 through 11.0.0-rc.3; 10.0.15 and 11.0.0-preview.2 work)

Database provider

Microsoft.EntityFrameworkCore.Sqlite 11.0.0. Also Microting.EntityFrameworkCore.MySql 11.0.0-preview.1 against MySQL 8.4.

Target framework

.NET 11.0

Operating system

macOS. Also Linux containers in production.

IDE

dotnet CLI, SDK 11.0.100