[BUG] `pylibcudf.Column.from_iterable_of_py(<nested_list>, dtype=plc.TypeId.LIST)` erroneously raises

Open Beginner friendly
#21,092 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
data

Research direction

Start in pylibcudf/column.pyx at Column.from_iterable_of_py and inspect the related DataType handling in pylibcudf/types.pyx. Reproduce the nested-list example with an explicit LIST dtype, then verify that it follows the same inferred inner-type behavior as the call without dtype and no longer raises NotImplementedError.

Written by the indexing model from the issue text.

Description

bug pylibcudf

Steps/Code to reproduce bug

In [1]: import pylibcudf as plc

In [2]: plc.Column.from_iterable_of_py([[1, 2, 3]])  # works
Out[2]: <pylibcudf.column.Column at 0x77d150dcbd60>

In [3]: plc.Column.from_iterable_of_py([[1, 2, 3]], dtype=plc.DataType(plc.TypeId.LIST))
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/cudf/python/pylibcudf/pylibcudf/types.pyx:151, in pylibcudf.types.DataType._python_typecode.__get__()

KeyError: <type_id.LIST: 24>

During handling of the above exception, another exception occurred:

NotImplementedError                       Traceback (most recent call last)
Cell In[3], line 1
----> 1 plc.Column.from_iterable_of_py([[1, 2, 3]], dtype=plc.DataType(plc.TypeId.LIST))

File ~/cudf/python/pylibcudf/pylibcudf/column.pyx:1238, in pylibcudf.column.Column.from_iterable_of_py()

File ~/cudf/python/pylibcudf/pylibcudf/types.pyx:153, in pylibcudf.types.DataType._python_typecode.__get__()

NotImplementedError: No Python typecode for DataType <type_id.LIST: 24>

I believe here we should still be "flattening" and inferring the inner type in the else clause if a plc.DataType(plc.TypeId.LIST) is passed

        if dtype is None:
            depth, py_dtype = _infer_list_depth_and_dtype(obj)
            dtype = DataType.from_py(py_dtype)
        else:
            depth, _ = _infer_list_depth_and_dtype(obj)

Expected behavior
Same as Out[2]

Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
2d 16h
Merged PRs (30d)
251

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/cudf

All issues in NVIDIA/cudf

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.