online-ml/river

Migrate all mini-batch (_many) methods to narwhals for dataframe-agnostic support

Aberta

#1.919 aberto em 24 de jun. de 2026

 (3 comentários) (0 reação) (0 responsável)Python (553 forks)batch import
EnhancementGood first issue

Métricas do repositório

Stars
 (4.574 estrelas)
Métricas de merge de PR
 (Mesclagem média 47d 6h) (46 fundiu PRs em 30d)

Description

Motivation

#1900 introduced dataframe-agnostic mini-batching via narwhals: inputs are wrapped at the method boundary, the numpy compute core stays untouched, and outputs are rebuilt in the caller's native backend (pandas / polars / pyarrow / nullable / arrow-backed pandas). The helpers live in river/utils/dataframe.py:

  • into_frame / into_series — wrap native inputs
  • to_numpy — extract a float64 array for the compute core
  • to_native_frame / to_native_series — rebuild output in the caller's backend, preserving the pandas index

So far only linear_model (GLM + LinearRegression, LogisticRegression, BayesianLinearRegression) uses this. Every other mini-batch method still hard-codes pandas (pd.DataFrame/pd.Series signatures, .values, .columns, etc.). The goal of this issue is to migrate all _many methods to narwhals so that any narwhals-supported backend works end to end.

Scope

Mini-batch methods to migrate (learn_many, predict_many, predict_proba_many, transform_many, and friends):

Base classes (signatures/type hints — do first, they're authoritative):

  • base/classifier.pyMiniBatchClassifier
  • base/regressor.pyMiniBatchRegressor
  • base/transformer.pyMiniBatchTransformer

Concrete estimators (still pandas-only):

Composition (should mostly fall out once the above are done, but need verifying):

Approach

For each method:

  1. Wrap inputs with into_frame / into_series at entry.
  2. Drop to numpy via to_numpy for the compute core (leave the math unchanged).
  3. Rebuild outputs with to_native_frame / to_native_series so the caller's backend and index are preserved.
  4. Replace pd.DataFrame / pd.Series type hints with the narwhals IntoDataFrame / IntoSeries aliases.
  5. Add multi-backend tests using the existing frame_backend fixture (river/conftest.py), mirroring test_glm.py.

learn_many/predict_many outputs must stay byte-for-byte identical for the pandas path so nothing regresses.

Notes / gotchas (from #1900)

  • to_numpy forces float64 coercion because pandas ArrowDtype columns otherwise come back as object and break downstream ufuncs.
  • Non-pandas backends require string column labels; to_native_frame stringifies for those.
  • The pandas index is only carried over when nw.maybe_get_index() is not None.

Guia do colaborador