rapidsai/cudf
View on GitHub[BUG] cudf astype('int32') behavior does not match pandas for strings with underscores
Open
#12,047 opened on Nov 2, 2022
Pythonbuggood first issue
Repository metrics
- Stars
- (6,000 stars)
- PR merge metrics
- (Avg merge 17d 21h) (230 merged PRs in 30d)
Description
Steps/Code to reproduce bug
$ python3.9 -m IPython
Python 3.9.14 (main, Sep 7 2022, 23:43:48)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import pandas as pd, cudf
In [2]: pd.__version__, cudf.__version__
Out[2]: ('1.4.4', '22.10.00a+392.g1558403753')
In [3]: pdf = pd.DataFrame({'a': ['123_1']})
In [4]: pdf.a.astype('int32')
Out[4]:
0 1231
Name: a, dtype: int32
In [5]: cdf = cudf.DataFrame({'a': ['123_1']})
In [6]: cdf.a.astype('int32')
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In [6], line 1
----> 1 cdf.a.astype('int32')
...
File ~/.local/lib/python3.9/site-packages/cudf/core/column/string.py:5314, in StringColumn.as_numerical_column(self, dtype, **kwargs)
5312 if out_dtype.kind in {"i", "u"}:
5313 if not libstrings.is_integer(string_col).all():
-> 5314 raise ValueError(
5315 "Could not convert strings to integer "
5316 "type due to presence of non-integer values."
5317 )
5318 elif out_dtype.kind == "f":
5319 if not libstrings.is_float(string_col).all():
ValueError: Could not convert strings to integer type due to presence of non-integer values.
Expected behavior
cudf to produce the same result as pandas
Environment overview
$ nvidia-smi
Wed Nov 2 16:48:53 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02 Driver Version: 510.85.02 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 56C P8 15W / 70W | 0MiB / 15360MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+