TargetEncoder reuses the first level's shrinkage factor for all levels when m > 0

Open Beginner friendly
  • Is something wrong?
#95 4 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@parth-patel010 is already working on this.

Since Nov 3, 2026.

  • #96 by @parth-patel010 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Domain
machine-learning

Research direction

Start in target_encoder_fit, especially the per-level loop inside the feature_mapper closure, and read the existing TargetEncoder tests. Add regression coverage for finite m and m = :auto, checking that each level gets its own shrinkage factor. Done means the reproduced level encodings match the per-level expected values and the existing tests still pass.

Written by the indexing model from the issue text.

Description

TargetEncoder with m > 0 (or m = :auto) is supposed to compute the shrinkage factor λ = n_i / (m + n_i) separately for every level, but only the first level of the first encoded column gets its own λ. Every other level (and every other column) reuses that same value.

To reproduce

using MLJBase, MLJTransforms, CategoricalArrays, Statistics

X = (A = categorical(["a", "a", "b", "b", "b", "b", "b", "b"]),)
y = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]

mach = fit!(machine(TargetEncoder(m = 2), X, y), verbosity = 0)
Xt = MLJBase.transform(mach, X)
println("encoded A: ", Xt.A)

prior = mean(y)
for lvl in ("a", "b")
    t = y[X.A .== lvl]
    λ = length(t) / (2 + length(t))
    println(lvl, ": n=", length(t), "  λ=", λ, "  expected=", λ * mean(t) + (1 - λ) * prior)
end

Output:

encoded A: [3.0, 3.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0]
a: n=2  λ=0.5  expected=3.0
b: n=6  λ=0.75  expected=5.25

Level "b" should be encoded as 0.75 * 5.5 + 0.25 * 4.5 = 5.25, but it gets 5.0, which is what you get with level "a"'s λ = 0.5. Same thing with m = :auto: the m estimated for the first level is reused for all the others (for the data above but with y = [1, 2, 3, 5, 8, 13, 21, 34], level "b" comes out as 10.8985 instead of 13.5697). Binary and multiclass targets are affected the same way.

Cause

In target_encoder_fit, the per-level loop inside the feature_mapper closure assigns back to the captured outer variables:

m == :auto && (m = compute_m_auto(task, targets_for_level; y_var = y_var))
lambda = compute_shrinkage(targets_for_level; m = m, λ = lambda)

After the first level, lambda is no longer 1.0, so compute_shrinkage treats it as a user-supplied λ and just returns it. With m = :auto, m is overwritten by a number on the first level so it is never re-estimated. The existing tests only use m = Inf, where λ is 0 for every level anyway, so they don't catch this.

Versions

  • Julia 1.13.3
  • MLJTransforms 0.2.1 (current dev, 1f79daf)
  • MLJBase 1.15.0, CategoricalArrays 1.1.1
Dominant language
Julia
Stars
1
Forks
5
Avg merge
1d 16h
Merged PRs (30d)
8

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JuliaAI/MLJTransforms.jl

All issues in JuliaAI/MLJTransforms.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.