rapidsai/cudf

[BUG] Python groupby rolling aggregations return index inconsistent with pandas

Open

#10,249 opened on Feb 8, 2022

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (735 forks)batch import
Pythonbugdaskgood first issue

Repository metrics

Stars
 (6,000 stars)
PR merge metrics
 (Avg merge 17d 21h) (230 merged PRs in 30d)

Description

Python groupby rolling aggregations return a single Index that corresponds to the original row position of the element, but in pandas return a MultiIndex that includes both the groupby key(s) and original row position.

This is not currently blocking any behavior with Dask + cuDF, as grouped rolling operations are blocked by #10173

import pandas as pd
import cudf
import numpy as np
​
df = cudf.datasets.randomdata(nrows=100000)
pdf = df.to_pandas()
​
print(pdf.groupby(['id']).rolling(window=3).x.mean().head())
print(df.groupby(['id']).rolling(window=3).x.mean().head())
id        
879  43605   NaN
881  3941    NaN
882  29855   NaN
884  14616   NaN
     70864   NaN
Name: x, dtype: float64
43605    <NA>
3941     <NA>
29855    <NA>
14616    <NA>
70864    <NA>
Name: x, dtype: float64

Contributor guide

Research direction
Investigate the code for groupby rolling in cuDF, specifically the function that computes the rolling aggregation, and modify it to return a MultiIndex with groupby keys, consistent with pandas.
Tech stack
python
Domain
data
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonpandasGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40