lensacom/sparkit-learn

[RFC] Integrate with numba / parakeet

Open

#26 opened on Jun 11, 2015

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Python (1,144 stars) (255 forks)batch import
enhancementhelp wantedrfc

Description

Execution time could benefit from an optimizer like numba.

From numba's documentation: "Numba works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically (using the included pycc tool). Numba supports compilation of Python to run on either CPU or GPU hardware, and is designed to integrate with the Python scientific software stack."

Contributor guide

Tech stack
pythonscikit learnsparknumpy
Domain
performancedatamachine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
unclear
Prerequisites
numba basicsPython and numpyPySparkscikit learn
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
10
Research direction
Investigate the sparkit learn codebase to identify performance bottlenecks, particularly in core computation functions like parallel predict or fit. Review numba documentation and examples for integrating with numpy and scikit learn. Propose specific functions to accelerate with numba and provide benchmark comparisons to demonstrate improvements.