rapidsai/cudf

[BUG] Add support for `force_ascii=False` when writing to JSON with cuDF engine

Open

#15,211 opened on Mar 1, 2024

View on GitHub
 (4 comments) (1 reaction) (0 assignees)C++ (6,000 stars) (735 forks)batch import
PythonbugcuIOgood first issuelibcudf

Description

Describe the bug Ideally, we should eventually support engine="cudf" and force_ascii=False together with to_json. For now, we should update the documentation and/or provide a warning for users.

Steps/Code to reproduce bug

import cudf

df = cudf.DataFrame({"a": [1,2,3], "b": ["4","5","🌱"]})
df.to_json("test.jsonl", orient="records", lines=True, engine="cudf", force_ascii=False)

produces a TypeError: write_json() got an unexpected keyword argument 'force_ascii'.

I can do a df.to_json("test.jsonl", orient="records", lines=True, force_ascii=False) and see the emoji in the .jsonl file, and I can also do a df.to_json("test.jsonl", orient="records", lines=True, engine="cudf") and see the emoji represented as "\ud83c\udf31" in the .jsonl file. But I am unable to see the emoji represented as is in the file, while also writing with the cuDF engine.

Environment details I tested this with the latest cuDF version.

Contributor guide

Tech stack
python
Domain
data
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
PythonBasic understanding of cuDF
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Investigate the `to json` method in cuDF, specifically the `write json` function for the cuDF engine. Look for where the `force ascii` parameter is handled. Check if there is existing documentation for `to json` and consider adding a warning or updating docs to clarify that `force ascii` is not yet supported with the cuDF engine. Refer to the issue comments for any additional insights.