rapidsai/cudf

[FEA] Implement find_valid_index and its permutations (first_ and last_)

Open

#1,480 opened on Apr 22, 2019

View on GitHub
 (4 comments) (0 reactions) (0 assignees)C++ (6,000 stars) (735 forks)batch import
Pythonfeature requestgood first issue

Description

Is your feature request related to a problem? Please describe. As a user, I'd like to be able to call last_valid_index or first_valid_index to get the index of the first/last non-null value in a Series.

Describe the solution you'd like I'd like to be able to do the following with cudf:

import pandas as pd
import numpy as np

ser1 = [1,2,3,4]
ser2 = [1,2,3,None]
pdf = pd.DataFrame({'a':ser1, 'b':ser2})
print(pdf.b.last_valid_index())
2

Additional context The pandas implementation is here. This will help with API compatibility.

Contributor guide

Tech stack
pythoncpppandas
Domain
dataapi
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic knowledge of pandas APIUnderstanding of C++ and CUDAFamiliarity with cudf codebase
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
First, examine the pandas implementation of last valid index and first valid index linked in the issue to understand the expected behavior. Then, explore the cudf codebase, particularly the Series class and its existing methods for null handling and index operations. Look for similar API compatibility implementations (e.g., how other methods like 'idxmax' were added). The key is to implement a CUDA kernel that scans the series from the specified direction to find the first/last non null index. Ensure the function returns the appropriate type (int or NaT) as per pandas. After implementation, test against pandas behavior using the provided example.