Performance benchmarking (fit)
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- julia, scikit-learn
- Domain
- machine-learning, performance, testing
Research direction
Start by defining a systematic benchmarking approach for the listed regression and classification methods, including function, gradient, and Hessian call counts and objective convergence. Compare the implementations against scikit-learn and quantreg, and consider the issue complete when the listed cases have reproducible measurements or convergence plots against the stated expectations.
Written by the indexing model from the issue text.
Description
Before starting this, need a way to systematically:
- trace number of function calls, number of gradient calls, number of hessian calls
- have a way to stop with universal criterion OR show a plot where the objective function decreases and eventually hits the same value as that from ref package.
Against scikitlearn
expect on par or better
- ridge (in big case should see improvements from using CG)
- analytical (should see no real diff)
- CG
- lasso
- FISTA
- ISTA
- elnet
- logistic (no or l2 penalty)
- (cannot test Newton)
- Newton CG see issue #24
- LBFGS
- logistic (elnet penalty)
- FISTA
- ISTA
- multinomial (no or l2 penalty)
- Newton CG see issue #24
- LBFGS
- multinomial (elnet penalty)
- FISTA
- ISTA
Against quantreg
expect a bit worse (quantreg is effectively in cpp)
- quantile regression
- Dominant language
- Julia
- Stars
- 86
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from JuliaAI/MLJLinearModels.jl
-
Difficulty 3/5 1-2 days Newbie friendliness 78/100
JuliaAI/MLJLinearModels.jl#187 · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
JuliaAI/MLJLinearModels.jl#180 ·
-
help wanted
Difficulty 5/5 Over a week Newbie friendliness 25/100
JuliaAI/MLJLinearModels.jl#163 · 4 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
JuliaAI/MLJLinearModels.jl#160 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
JuliaAI/MLJLinearModels.jl#159 · 3 comments ·
All issues in JuliaAI/MLJLinearModels.jl
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
Sienna-Platform/PowerSimulations.jl#1737 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
FluxML/Optimisers.jl#254 ·
-
Minor: floodplain warning percentage, scalar output error message and a never-true identity checkOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
JuliaAI/MLJTransforms.jl#101 ·
Maintainers usually reply within 1 day
-
@contents blocks fail to render on Julia 1.14Possibly taken @sbfnk claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
LuxDL/DocumenterVitepress.jl#410 ·