scikit-learn/scikit-learn

Ability to cache FeatureUnion transformers

Open

#9,008 opened on Jun 6, 2017

View on GitHub
 (10 comments) (3 reactions) (0 assignees)Python (66,084 stars) (27,020 forks)batch import
Enhancementhelp wantedmodule:pipeline

Description

It seems reasonable to support a memory parameter to FeatureUnion like was recently added to Pipeline (#7990). It is valuable in the sense that parameters in some constituent transformers can be searched over while others are unchanged; those that are unchanged should not need to be re-fit from scratch.

Contributor guide

Tech stack
python
Domain
machine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
understanding of scikit learn Pipelineknowledge of FeatureUnion
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
Start by examining the existing `memory` parameter implementation in `Pipeline` class (see PR #7990 and sklearn/pipeline.py). Then understand how `FeatureUnion` currently handles sub transformers and where caching could be integrated. The implementation should mirror the `Pipeline` approach, likely modifying `FeatureUnion.fit` and `FeatureUnion.transform` to cache fitted transformers. Reference the `memory` keyword in `Pipeline` for guidance on joblib caching.