Weights in pipelines

Open
#1,005 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
julia

Research direction

Start at the machine(..., weights) and fit! entry points shown in the issue, comparing direct-model handling with Pipeline behavior. Trace how unsupported weights are reported in each path and identify the existing tests covering weighted fitting. Done means unsupported weights are handled consistently in direct models and pipelines, with tests confirming the expected failure behavior.

Written by the indexing model from the issue text.

Description

Hello,

I am trying to use weights in MLJ and surprised by the behaviour in a pipeline which I think is a bug. In the following, the supervised component does not support weights but neither an error is thrown nor a warning displayed.

using MLJBase
using MLJModels
using MLJLinearModels

n = 100
X = (A=categorical(rand([0, 1], n)), B=categorical(rand([0, 1], n)))
y = categorical(rand([0, 1], n))
weights = [y_ == true ? 0.1 : 0.8 for y_ in y]
pipe = Pipeline(MLJModels.ContinuousEncoder(), LogisticClassifier())

# Train without weights: all good
unweighted_mach = machine(pipe, X, y)
fit!(unweighted_mach, verbosity=2)
preds_unweighted = predict(unweighted_mach)

# Train with weights: no warning, no error and weights are ignored
weighted_mach = machine(pipe, X, y, weights)
fit!(weighted_mach, verbosity=2)
preds_weighted = predict(weighted_mach)

[x.prob_given_ref[1] for x in preds_unweighted] == [x.prob_given_ref[1] for x in preds_weighted] # returns true

More generally, outside a pipeline, passing weights to models that don't support it only prints a warning and fit proceeds. I thought it would throw an error:

n = 100
X = (A=rand(n), B=rand(n))
y = categorical(rand([0, 1], n))
weights = [y_ == true ? 0.1 : 0.8 for y_ in y]

weighted_mach = machine(LogisticClassifier(), X, y, weights) # warning here
fit!(weighted_mach)
predict(weighted_mach)
Dominant language
Julia
Stars
164
Forks
46
Avg merge
23h 18m
Merged PRs (30d)
3

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaAI/MLJBase.jl

All issues in JuliaAI/MLJBase.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.