AzureDocumentDB: CosineDistance returns similarity scores, and ScoreThreshold filters the wrong way

Fermée
  • Quelque chose ne va pas ?
#72 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Une pull request liée a déjà été fusionnée.

  • #78 par @Laurianti — fusionnée

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
45/100
Type d'issue
Bug
Clarté
Plutôt claire
Activité
Active
Stack technique
azure, csharp, docker, mongodb
Domaine
databases, testing-qa

Piste de recherche

Read MEVD/src/AzureDocumentDB/DocumentDBCollectionCreateMapping.cs, MEVD/src/AzureDocumentDB/DocumentDBCollectionSearchMapping.cs, and MEVD/src/AzureDocumentDB/DocumentDBCollection.cs to compare the provider’s cosine score and threshold semantics with InMemory. Run the supplied dotnet run repro.cs, then inspect MEVD/test/AzureDocumentDB.ConformanceTests/DocumentDBDistanceFunctionTests.cs for the current score assertions. Completion requires agreeing whether to normalize similarity into distance, adjusting the threshold comparison, and updating conformance coverage without silently changing existing behavior.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

bug

With CosineDistance, the default, AzureDocumentDB returns the cosine similarity as Score, and ScoreThreshold keeps scores >= the threshold. InMemory returns the distance and keeps scores <= it.

Repro: with Docker running, dotnet run repro.cs.

#:package [email protected]
#:package [email protected]
#:package [email protected]
#:property PublishAot=false

using System.Security.Authentication;
using CommunityToolkit.VectorData.AzureDocumentDB;
using CommunityToolkit.VectorData.InMemory;
using DotNet.Testcontainers.Builders;
using Microsoft.Extensions.VectorData;
using MongoDB.Driver;

// Starts DocumentDB local in Docker, then compares InMemory and AzureDocumentDB with CosineDistance.
await using var container = new ContainerBuilder("ghcr.io/microsoft/documentdb/documentdb-local:latest")
    .WithEnvironment("USERNAME", "user")
    .WithEnvironment("PASSWORD", "Pass123!")
    .WithPortBinding(10260, true)
    .WithWaitStrategy(Wait.ForUnixContainer().UntilInternalTcpPortIsAvailable(10260))
    .Build();
await container.StartAsync();

var settings = MongoClientSettings.FromConnectionString($"mongodb://user:Pass123!@{container.Hostname}:{container.GetMappedPublicPort(10260)}/?tls=true");
settings.SslSettings = new SslSettings { EnabledSslProtocols = SslProtocols.Tls12, ServerCertificateValidationCallback = (_, _, _, _) => true };
var documentDb = new DocumentDBVectorStore(new MongoClient(settings).GetDatabase("repro"));
var inMemory = new InMemoryVectorStore();

Hotel[] hotels =
[
    new() { Id = "same", Vector = new float[] { 1, 0, 0 } },
    new() { Id = "orthogonal", Vector = new float[] { 0, 1, 0 } },
    new() { Id = "opposite", Vector = new float[] { -1, 0, 0 } },
];
ReadOnlyMemory<float> query = new float[] { 1, 0, 0 };

foreach (var (name, store) in new (string, VectorStore)[] { ("InMemory", inMemory), ("DocumentDB", documentDb) })
{
    var collection = store.GetCollection<string, Hotel>("hotels");
    await collection.EnsureCollectionDeletedAsync();
    await collection.EnsureCollectionExistsAsync();
    await collection.UpsertAsync(hotels);
    await Task.Delay(1000);

    Console.WriteLine($"{name} CosineDistance");
    await foreach (var r in collection.SearchAsync(query, 3))
        Console.WriteLine($"  {r.Record.Id,-10} score {r.Score:0.##}");
    var within = await collection.SearchAsync(query, 3, new() { ScoreThreshold = 1.5 }).Select(r => r.Record.Id).ToListAsync();
    Console.WriteLine($"  ScoreThreshold = 1.5 -> [{string.Join(", ", within)}]");
}

public sealed class Hotel
{
    [VectorStoreKey] public string Id { get; set; } = "";
    [VectorStoreVector(3, DistanceFunction = DistanceFunction.CosineDistance)] public ReadOnlyMemory<float> Vector { get; set; }
}

InMemory CosineDistance
  same       score 0
  orthogonal score 1
  opposite   score 2
  ScoreThreshold = 1.5 -> [same, orthogonal]
DocumentDB CosineDistance
  same       score 1
  orthogonal score 0
  opposite   score -1
  ScoreThreshold = 1.5 -> []

Cause:

Returning 1 - similarity and filtering with $lte would match the other providers, but it changes the scores existing users get. Which way do you prefer? I can send the PR.

Langage dominant
C#
Étoiles
14
Forks
9
Merge moyen
1 j 23 h
PR mergées (30 j)
13

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de CommunityToolkit/AI

Toutes les issues de CommunityToolkit/AI

Issues similaires

Plus d'issues C#

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.