scikit-learn/scikit-learn

Investigate SAG/SAGA solver

Open

#23,180 opened on Apr 21, 2022

View on GitHub
 (9 comments) (0 reactions) (0 assignees)Python (27,020 forks)batch import
BugModeratehelp wantedmodule:linear_model

Repository metrics

Stars
 (66,084 stars)
PR merge metrics
 (Avg merge 10d) (90 merged PRs in 30d)

Description

Description

The newly introduced tight tests for Ridge in #22910 together with the random seed fixture in #22749 revealed some shortcomings of the sag and saga solver, some are already mentioned in #21305:

  1. It shows some random behavior even with fixed random seed.
  2. The tol needs to be set much smaller to receive comparable results with the other solvers of Ridge.
  3. Saga seems more stable on dense data X, and often needs much more iterations on sparse X.
  4. fit_intercept=True seems to converge slower.

Ideally, the cause for both issues can be identified and fixed.

Some links for context

https://github.com/scikit-learn/scikit-learn/issues/23014 https://github.com/scikit-learn/scikit-learn/pull/23017 https://github.com/scikit-learn/scikit-learn/pull/23026 https://github.com/scikit-learn/scikit-learn/pull/23152 #23177 #25198

Contributor guide

Research direction
Examine the numerical stability of SAG/SAGA solvers, especially with sparse data and fit intercept=True. Compare convergence behavior against other solvers and trace the implementation to identify bottlenecks or numerical issues.
Tech stack
python
Domain
machine learning
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30