rapidsai/cudf

[FEA] Pass column indices as `index_col` in `read_csv`

Open

#15,127 opened on Feb 23, 2024

View on GitHub
 (7 comments) (0 reactions) (0 assignees)C++ (6,000 stars) (735 forks)batch import
0 - BacklogPythonfeature requestgood first issue

Description

If I want to use the index_col parameter to set certain columns as indices when reading a csv file, I cannot pass a list of column indices (like in pandas). I can pass a list of column labels though:

cudf.read_csv(filepath, index_col=[0])
KeyError: 'None of [0] are in the columns'

cudf.read_csv(filepath, index_col=['family'])

While this is not a huge issue, I imagine the following is a common scenario: You have know that the first 3 columns are index columns, but you don't exactly know how each are spelt ('date' vs 'Date' etc.). In this case, if passing a list of column indices were possible, index_col=[0,1,2] would have worked fine; otherwise, you will have to read the file without specifying index columns and set index later (or require trial and error to guess the column labels).

Is it possible for index_col to accept list of indices like in pandas?

Contributor guide

Tech stack
python
Domain
data
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
pandas knowledgePythoncudf basicsCSV file handling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Investigate the `index col` parameter in cudf's `read csv` function, located in the Python source (e.g., `python/cudf/cudf/ lib/csv.pyx`). Determine how column labels are currently validated and how to extend to accept integer indices. Ensure behavior matches pandas: integers refer to column positions. Add error handling and tests.