Several translations declare the wrong nullability, so `!=`, `!(==)` and `== null` drop or keep the wrong rows (same class as #3585)

Aperta
  • Qualcosa non va?
#4,084 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp, postgresql
Ambito
databases

Direzione di ricerca

The issue points to NpgsqlRangeTranslator.cs, NpgsqlNodaTimeMemberTranslatorPlugin.cs, NpgsqlDateTimeMemberTranslator.cs, NpgsqlQueryableMethodTranslatingExpressionVisitor.cs, NpgsqlLTreeTranslator.cs, NpgsqlNetTopologySuiteMemberTranslatorPlugin.cs and NpgsqlSqlNullabilityProcessor.cs where nullability is misdeclared. Start by reading #3999's fix for lower()/upper(), then adjust argumentsPropagateNullability to [false] for functions returning NULL on non-NULL input and mark array ||/array_remove as non-strict. Done when the repro LINQ queries return matching .NET and database row counts.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Environment
PostgreSQL 18.6 (postgres:18); the NTS part on 18.6 with PostGIS 3.6.4 and GEOS 3.14.1 (postgis/postgis:18-3.6), session TimeZone Etc/UTC
.NET 10 runtime 10.0.11, Npgsql.EntityFrameworkCore.PostgreSQL(.NodaTime/.NetTopologySuite) 10.0.3, EF Core 10.0.4
.NET 11 runtime 11.0.0-rc.2, Npgsql.EntityFrameworkCore.PostgreSQL(.NodaTime/.NetTopologySuite) 11.0.0-rc.2, EF Core 11.0.0-rc.2
Also Npgsql 10.0.3 on both lines, NodaTime 3.2.2, NetTopologySuite 2.6.0, Npgsql.NetTopologySuite 10.0.3, macOS arm64

Both lines print the same output. main at 150011d has the same code.

#3585 (fixed in #3999) was about NodaTime Interval.Start/End. lower() and upper() were declared with argumentsPropagateNullability: true, so EF rewrote lower(x) IS NULL to x IS NULL and dropped it for a required column. The same declaration is still used for other functions that return NULL for a non-NULL input. The opposite mistake exists for functions and operators that return a value for a NULL input. Either way, EF's compensation for C# null semantics in != and !(==) disappears, or == null folds to a constant.

LINQ Row .NET PostgreSQL SQL
r.Range.LowerBound != 5 (,10) match no match lower(r."Range") <> 5
r.Date.Month != 6 LocalDate.MaxIsoValue, written as infinity match no match date_part('month', r."Date")::int <> 6
r.Timestamp.Month != 6 DateTime.MaxValue, written as infinity match no match date_part('month', r."Timestamp" AT TIME ZONE 'UTC')::int <> 6
r.Tags.Where(t => t != r.Name).Count() != 2 {a,b}, Name NULL no match match cardinality(array_remove(r."Tags", r."Name")) <> 2 OR r."Name" IS NULL
r.NullableInts.Concat(r.Ints).Count() == 2 and != 2 NULL, {7,8} (throws) both match cardinality(r."NullableInts" || r."Ints") <> 2 OR r."NullableInts" IS NULL
(LTree?)LTree.LongestCommonAncestor(r.A, r.B) == null 'a.b', '': lca() is NULL (no .NET implementation) no match WHERE FALSE
r.Point.Z != 0 POINT (1 2): NTS gives NaN, ST_Z NULL match no match ST_Z(r."Point") <> 0.0
r.Line.StartPoint == null LINESTRING EMPTY: null on both sides match no match WHERE FALSE
Repro (ranges, date parts, arrays, LTree) 
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using NodaTime;
using NpgsqlTypes;

await using var context = new TestContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

context.Rows.Add(new Row
{
    Range = new NpgsqlRange<int>(0, false, true, 10, false, false), // (,10)
    Date = LocalDate.MaxIsoValue,                                   // written as 'infinity'
    Timestamp = DateTime.MaxValue,                                  // written as 'infinity'
    Tags = ["a", "b"],
    Name = null,
    Ints = [7, 8],
    NullableInts = null,
    A = new LTree("a.b"),
    B = new LTree(""),
});
await context.SaveChangesAsync();
var rows = await context.Rows.AsNoTracking().ToListAsync();

async Task Compare(string label, Expression<Func<Row, bool>> predicate)
    => Console.WriteLine($"{label}: .NET {rows.Count(predicate.Compile())}, database {await context.Rows.CountAsync(predicate)}");

await Compare("Range.LowerBound != 5", r => r.Range.LowerBound != 5);
await Compare("Date.Month != 6", r => r.Date.Month != 6);
await Compare("Timestamp.Month != 6", r => r.Timestamp.Month != 6);
await Compare("Tags.Where(t => t != Name).Count() != 2", r => r.Tags.Where(t => t != r.Name).Count() != 2);

// .NET throws on null.Concat; PostgreSQL's NULL || '{7,8}' is '{7,8}', yet the row matches both:
Console.WriteLine($"NullableInts.Concat(Ints).Count() == 2: database {await context.Rows.CountAsync(r => r.NullableInts!.Concat(r.Ints).Count() == 2)}");
Console.WriteLine($"NullableInts.Concat(Ints).Count() != 2: database {await context.Rows.CountAsync(r => r.NullableInts!.Concat(r.Ints).Count() != 2)}");

// lca() of 'a.b' and '' is NULL in PostgreSQL:
Console.WriteLine($"LongestCommonAncestor(A, B) projected: {await context.Rows.Select(r => (string?)LTree.LongestCommonAncestor(r.A, r.B)).SingleAsync() ?? "NULL"}");
Console.WriteLine($"LongestCommonAncestor(A, B) == null: database {await context.Rows.CountAsync(r => (LTree?)LTree.LongestCommonAncestor(r.A, r.B) == null)}");

public class TestContext : DbContext
{
    public DbSet<Row> Rows => Set<Row>();

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseNpgsql(Environment.GetEnvironmentVariable("Test__Npgsql__DefaultConnection"), o => o.UseNodaTime())
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
        => modelBuilder.HasPostgresExtension("ltree");
}

public class Row
{
    public int Id { get; set; }
    public NpgsqlRange<int> Range { get; set; }
    public LocalDate Date { get; set; }
    public DateTime Timestamp { get; set; }
    public string[] Tags { get; set; } = [];
    public string? Name { get; set; }
    public int[] Ints { get; set; } = [];
    public int[]? NullableInts { get; set; }
    public LTree A { get; set; }
    public LTree B { get; set; }
}

Output, the same on both lines:

Range.LowerBound != 5: .NET 1, database 0
Date.Month != 6: .NET 1, database 0
Timestamp.Month != 6: .NET 1, database 0
Tags.Where(t => t != Name).Count() != 2: .NET 0, database 1
NullableInts.Concat(Ints).Count() == 2: database 1
NullableInts.Concat(Ints).Count() != 2: database 1
LongestCommonAncestor(A, B) projected: NULL
LongestCommonAncestor(A, B) == null: database 0
Repro (NetTopologySuite) 
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using NetTopologySuite.Geometries;

await using var context = new TestContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

context.Rows.Add(new Row { Point = new Point(1, 2), Line = LineString.Empty });
await context.SaveChangesAsync();
var rows = await context.Rows.AsNoTracking().ToListAsync();

async Task Compare(string label, Expression<Func<Row, bool>> predicate)
    => Console.WriteLine($"{label}: .NET {rows.Count(predicate.Compile())}, database {await context.Rows.CountAsync(predicate)}");

await Compare("Point.Z != 0 (2D point: NTS NaN, ST_Z NULL)", r => r.Point.Z != 0);
await Compare("Line.StartPoint == null (empty line: both null)", r => r.Line.StartPoint == null);

public class TestContext : DbContext
{
    public DbSet<Row> Rows => Set<Row>();

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseNpgsql(Environment.GetEnvironmentVariable("Test__Npgsql__DefaultConnection"), o => o.UseNetTopologySuite())
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
        => modelBuilder.HasPostgresExtension("postgis");
}

public class Row
{
    public int Id { get; set; }
    public Point Point { get; set; }
    public LineString Line { get; set; }
}

Output, the same on both lines:

Point.Z != 0 (2D point: NTS NaN, ST_Z NULL): .NET 1, database 0
Line.StartPoint == null (empty line: both null): .NET 1, database 0

Where each one comes from:

A possible fix is the one #3999 made: argumentsPropagateNullability: [false] for functions that can return NULL for non-NULL input, and treating array_remove and array || as not strict.

Lingua principale
C#
Stelle
1.8k
Fork
268
Merge medio
2h 35m
PR unite (30g)
136

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di npgsql/efcore.pg

Tutte le issue di npgsql/efcore.pg

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.