online-ml/river

Migrate all mini-batch (_many) methods to narwhals for dataframe-agnostic support

Aperta

#1919 aperta il 24 giu 2026

 (3 commenti) (0 reazioni) (0 assegnatari)Python (553 fork)batch import
EnhancementGood first issue

Metriche repository

Star
 (4574 stelle)
Metriche merge PR
 (Merge medio 47g 6h) (46 PR mergiate in 30 g)

Descrizione

Motivation

#1900 introduced dataframe-agnostic mini-batching via narwhals: inputs are wrapped at the method boundary, the numpy compute core stays untouched, and outputs are rebuilt in the caller's native backend (pandas / polars / pyarrow / nullable / arrow-backed pandas). The helpers live in river/utils/dataframe.py:

  • into_frame / into_series — wrap native inputs
  • to_numpy — extract a float64 array for the compute core
  • to_native_frame / to_native_series — rebuild output in the caller's backend, preserving the pandas index

So far only linear_model (GLM + LinearRegression, LogisticRegression, BayesianLinearRegression) uses this. Every other mini-batch method still hard-codes pandas (pd.DataFrame/pd.Series signatures, .values, .columns, etc.). The goal of this issue is to migrate all _many methods to narwhals so that any narwhals-supported backend works end to end.

Scope

Mini-batch methods to migrate (learn_many, predict_many, predict_proba_many, transform_many, and friends):

Base classes (signatures/type hints — do first, they're authoritative):

  • base/classifier.pyMiniBatchClassifier
  • base/regressor.pyMiniBatchRegressor
  • base/transformer.pyMiniBatchTransformer

Concrete estimators (still pandas-only):

Composition (should mostly fall out once the above are done, but need verifying):

Approach

For each method:

  1. Wrap inputs with into_frame / into_series at entry.
  2. Drop to numpy via to_numpy for the compute core (leave the math unchanged).
  3. Rebuild outputs with to_native_frame / to_native_series so the caller's backend and index are preserved.
  4. Replace pd.DataFrame / pd.Series type hints with the narwhals IntoDataFrame / IntoSeries aliases.
  5. Add multi-backend tests using the existing frame_backend fixture (river/conftest.py), mirroring test_glm.py.

learn_many/predict_many outputs must stay byte-for-byte identical for the pandas path so nothing regresses.

Notes / gotchas (from #1900)

  • to_numpy forces float64 coercion because pandas ArrowDtype columns otherwise come back as object and break downstream ufuncs.
  • Non-pandas backends require string column labels; to_native_frame stringifies for those.
  • The pandas index is only carried over when nw.maybe_get_index() is not None.

Guida contributor