Migrate all mini-batch (_many) methods to narwhals for dataframe-agnostic support
#1 919 ouverte le 24 juin 2026
Métriques du dépôt
- Stars
- (4 574 étoiles)
- Métriques de merge PR
- (Merge moyen 47j 6h) (46 PRs mergées en 30 j)
Description
Motivation
#1900 introduced dataframe-agnostic mini-batching via narwhals: inputs are wrapped at the method boundary, the numpy compute core stays untouched, and outputs are rebuilt in the caller's native backend (pandas / polars / pyarrow / nullable / arrow-backed pandas). The helpers live in river/utils/dataframe.py:
into_frame/into_series— wrap native inputsto_numpy— extract a float64 array for the compute coreto_native_frame/to_native_series— rebuild output in the caller's backend, preserving the pandas index
So far only linear_model (GLM + LinearRegression, LogisticRegression, BayesianLinearRegression) uses this. Every other mini-batch method still hard-codes pandas (pd.DataFrame/pd.Series signatures, .values, .columns, etc.). The goal of this issue is to migrate all _many methods to narwhals so that any narwhals-supported backend works end to end.
Scope
Mini-batch methods to migrate (learn_many, predict_many, predict_proba_many, transform_many, and friends):
Base classes (signatures/type hints — do first, they're authoritative):
-
base/classifier.py—MiniBatchClassifier -
base/regressor.py—MiniBatchRegressor -
base/transformer.py—MiniBatchTransformer
Concrete estimators (still pandas-only):
-
preprocessing/scale.py—StandardScaler& co. (.values/.columns) https://github.com/online-ml/river/pull/1932 -
naive_bayes/base.py—BaseNB(joint_log_likelihood_many,predict_proba_many, pandas.subtract) -
multiclass/ovr.py—OneVsResthttps://github.com/online-ml/river/pull/1933 -
anomaly/lof.py—LocalOutlierFactorhttps://github.com/online-ml/river/pull/1929 -
anomaly/svm.py—OneClassSVMhttps://github.com/online-ml/river/pull/1930 -
feature_extraction/vectorize.pyhttps://github.com/online-ml/river/pull/1904 -
preprocessing/ordinal.py—OrdinalEncoderhttps://github.com/online-ml/river/pull/1921 -
preprocessing/one_hot.py—OneHotEncoderhttps://github.com/online-ml/river/pull/1926 -
covariance/emp.py—EmpiricalCovariance(andEmpiricalPrecision) https://github.com/online-ml/river/pull/1923
Composition (should mostly fall out once the above are done, but need verifying):
-
compose/pipeline.py—Pipelinehttps://github.com/online-ml/river/pull/1932 -
compose/union.py—TransformerUnionhttps://github.com/online-ml/river/pull/1932 -
compose/func.py—FuncTransformerhttps://github.com/online-ml/river/pull/1932 -
compose/product.py—TransformerProducthttps://github.com/online-ml/river/pull/1932 -
compose/select.py—Selecthttps://github.com/online-ml/river/pull/1932
Approach
For each method:
- Wrap inputs with
into_frame/into_seriesat entry. - Drop to numpy via
to_numpyfor the compute core (leave the math unchanged). - Rebuild outputs with
to_native_frame/to_native_seriesso the caller's backend and index are preserved. - Replace
pd.DataFrame/pd.Seriestype hints with the narwhalsIntoDataFrame/IntoSeriesaliases. - Add multi-backend tests using the existing
frame_backendfixture (river/conftest.py), mirroringtest_glm.py.
learn_many/predict_many outputs must stay byte-for-byte identical for the pandas path so nothing regresses.
Notes / gotchas (from #1900)
to_numpyforces float64 coercion because pandasArrowDtypecolumns otherwise come back asobjectand break downstream ufuncs.- Non-pandas backends require string column labels;
to_native_framestringifies for those. - The pandas index is only carried over when
nw.maybe_get_index()is notNone.