rapidsai/cudf

[BUG] cudf astype('int32') behavior does not match pandas for strings with underscores

Open

#12,047 opened on Nov 2, 2022

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (735 forks)batch import
Pythonbuggood first issue

Repository metrics

Stars
 (6,000 stars)
PR merge metrics
 (Avg merge 17d 21h) (230 merged PRs in 30d)

Description

Steps/Code to reproduce bug

$ python3.9 -m IPython
Python 3.9.14 (main, Sep  7 2022, 23:43:48) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd, cudf

In [2]: pd.__version__, cudf.__version__
Out[2]: ('1.4.4', '22.10.00a+392.g1558403753')

In [3]: pdf = pd.DataFrame({'a': ['123_1']})

In [4]: pdf.a.astype('int32')
Out[4]: 
0    1231
Name: a, dtype: int32

In [5]: cdf = cudf.DataFrame({'a': ['123_1']})

In [6]: cdf.a.astype('int32')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [6], line 1
----> 1 cdf.a.astype('int32')

...

File ~/.local/lib/python3.9/site-packages/cudf/core/column/string.py:5314, in StringColumn.as_numerical_column(self, dtype, **kwargs)
   5312 if out_dtype.kind in {"i", "u"}:
   5313     if not libstrings.is_integer(string_col).all():
-> 5314         raise ValueError(
   5315             "Could not convert strings to integer "
   5316             "type due to presence of non-integer values."
   5317         )
   5318 elif out_dtype.kind == "f":
   5319     if not libstrings.is_float(string_col).all():

ValueError: Could not convert strings to integer type due to presence of non-integer values.

Expected behavior cudf to produce the same result as pandas

Environment overview

$ nvidia-smi 
Wed Nov  2 16:48:53 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02    Driver Version: 510.85.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   56C    P8    15W /  70W |      0MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Contributor guide