dask/dask

`.str.split` coerces `dtype` to `object`

Open

#11,884 opened on Apr 11, 2025

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Python (11,520 stars) (1,658 forks)batch import
dataframegood first issue

Description

Describe the issue:

The .str.split method coerces dtype from string to object. This behavior is also inconsistent with pandas (if that matters).

Minimal complete verifiable example:

Code

import dask.dataframe as dd
import pandas as pd

data = {"c": ["a,b,c", "d,e,f", "g,h,i"]}

# `pandas`
df = pd.DataFrame(data, dtype="string[pyarrow]")
print(df.dtypes)
print(df["c"].str.split(",", n=1, expand=True).dtypes)

# `dask`
ddf = dd.from_pandas(df)
print(ddf.dtypes)
print(ddf["c"].str.split(",", n=1, expand=True).dtypes)

Output

c    string[pyarrow]
dtype: object
0    string[pyarrow]
1    string[pyarrow]
dtype: object
c    string[pyarrow]
dtype: object
0    object
1    object
dtype: object

Environment:

  • Dask version: 2025.3.0
  • Python version: 3.10.16
  • Operating System: Ubuntu 24.04.2 LTS
  • Install method (conda, pip, source): pip

Contributor guide

Tech stack
python
Domain
data
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
basic Pythonfamiliarity with dask DataFrameunderstanding of pandas dtypes
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate the `.str.split` method in dask's StringAccessor (likely in `dask/dataframe/accessor.py`). The issue shows that when expand=True with n=1, the output columns have object dtype instead of preserving the original string[pyarrow] dtype. Compare with pandas' implementation to see how it preserves dtype. The fix likely involves ensuring the output Series/DataFrame uses the same dtype as the input column. Check if there's a meta or dtype propagation issue in the underlying operations.