AutoML BinaryClassificationExperiment ignores ColumnInformation.ExampleWeightColumnName — trainers are fitted unweighted

Offen
  • Stimmt etwas nicht?
#7,823 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Anfängerfreundlichkeit
70/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
csharp
Bereich
machine-learning

Rechercherichtung

Start at BinaryClassificationExperiment.Execute and trace how ColumnInformation.ExampleWeightColumnName is passed to the trainers and validation metrics; the issue names no source files or tests. Compare the AutoML result with the direct LbfgsLogisticRegression example using the supplied weighted fixture. Done when the experiment either applies weights to fitting and validation or clearly warns or refuses a weighted request.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

area-AutoML untriaged
System information
  • Microsoft.ML 5.0.0, Microsoft.ML.AutoML 0.23.0
  • .NET 10 (net10.0), macOS arm64
Describe the bug

ColumnInformation.ExampleWeightColumnName is accepted by BinaryClassificationExperiment.Execute, but the weight column does not reach the trainers the experiment builds, nor its validation metrics. No warning is emitted, so the caller gets an unweighted model while believing it is weighted.

To reproduce

Fixture: 1,000 X values in [0,1] (plus a noise column Y), each row written twice — once Label=true, once Label=false — with weight W = 10 / 1 for true / false when X > 0.5, and 1 / 10 otherwise. Unweighted, the label is independent of X (P = 0.5 everywhere); weighted, P(true | X=0.9) ≈ 0.9.

  1. AutoML (15 s budget):

    ml.Auto().CreateBinaryClassificationExperiment(15).Execute(train, new ColumnInformation
{
    LabelColumnName = "Label",
    ExampleWeightColumnName = "W",
    NumericColumnNames = { "X", "Y" },
});

    Best run LbfgsLogisticRegressionBinary: P(true | X=0.9) = 0.49, P(true | X=0.1) = 0.49 — identical to the same experiment with W placed in IgnoredColumnNames. The per-trainer best validation AUCs were also identical in both runs (FastTree 0.41, FastForest 0.40, SdcaLogisticRegression 0.50, LbfgsLogisticRegression 0.53).

  2. The same trainer fitted directly:

    ml.Transforms.Concatenate("Features", "X", "Y")
  .Append(ml.BinaryClassification.Trainers.LbfgsLogisticRegression(exampleWeightColumnName: "W"));

    gives 0.96 / 0.04.

Expected behavior

Either the trainers AutoML constructs receive exampleWeightColumnName (the binary, multiclass and regression trainers it sweeps all accept it), with validation metrics weighted the same way — or Execute refuses / warns when ExampleWeightColumnName is set, so a weighted request cannot silently produce an unweighted fit.

Why it matters

Labels collected by importance sampling, or from sources of differing reliability, are trained with per-row weights. A silently unweighted fit learns the sampling bias, and nothing in the result tells the caller.

Related: #6708 (documentation of ExampleWeightColumnName).

Vorherrschende Sprache
C#
Sterne
9.4k
Forks
2k
Ø Merge
2 T. 18 Std.
Gemergte PRs (30 T.)
64

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus dotnet/machinelearning

Alle Issues in dotnet/machinelearning

Ähnliche Issues

Weitere Issues zu C#

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.