online-ml/river

Migrate all mini-batch (_many) methods to narwhals for dataframe-agnostic support

オープン

#1,919 opened on 2026/06/24

 (3 件のコメント) (0 件のリアクション) (0 人の担当者)Python (553 件のフォーク)batch import
EnhancementGood first issue

Repository metrics

Stars
 (4,574 個のスター)
PR merge metrics
 (平均マージ 47d 6h) (30d で 46 merged PRs)

説明

Motivation

#1900 introduced dataframe-agnostic mini-batching via narwhals: inputs are wrapped at the method boundary, the numpy compute core stays untouched, and outputs are rebuilt in the caller's native backend (pandas / polars / pyarrow / nullable / arrow-backed pandas). The helpers live in river/utils/dataframe.py:

  • into_frame / into_series — wrap native inputs
  • to_numpy — extract a float64 array for the compute core
  • to_native_frame / to_native_series — rebuild output in the caller's backend, preserving the pandas index

So far only linear_model (GLM + LinearRegression, LogisticRegression, BayesianLinearRegression) uses this. Every other mini-batch method still hard-codes pandas (pd.DataFrame/pd.Series signatures, .values, .columns, etc.). The goal of this issue is to migrate all _many methods to narwhals so that any narwhals-supported backend works end to end.

Scope

Mini-batch methods to migrate (learn_many, predict_many, predict_proba_many, transform_many, and friends):

Base classes (signatures/type hints — do first, they're authoritative):

  • base/classifier.pyMiniBatchClassifier
  • base/regressor.pyMiniBatchRegressor
  • base/transformer.pyMiniBatchTransformer

Concrete estimators (still pandas-only):

Composition (should mostly fall out once the above are done, but need verifying):

Approach

For each method:

  1. Wrap inputs with into_frame / into_series at entry.
  2. Drop to numpy via to_numpy for the compute core (leave the math unchanged).
  3. Rebuild outputs with to_native_frame / to_native_series so the caller's backend and index are preserved.
  4. Replace pd.DataFrame / pd.Series type hints with the narwhals IntoDataFrame / IntoSeries aliases.
  5. Add multi-backend tests using the existing frame_backend fixture (river/conftest.py), mirroring test_glm.py.

learn_many/predict_many outputs must stay byte-for-byte identical for the pandas path so nothing regresses.

Notes / gotchas (from #1900)

  • to_numpy forces float64 coercion because pandas ArrowDtype columns otherwise come back as object and break downstream ufuncs.
  • Non-pandas backends require string column labels; to_native_frame stringifies for those.
  • The pandas index is only carried over when nw.maybe_get_index() is not None.

コントリビューターガイド