`Dataset.to_sql` doesn't utilize `num_proc`
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
Research direction
Start at Dataset.to_sql() and compare its arguments with SqlDatasetWriter in src/datasets/io/sql.py, especially the num_proc argument noted in the issue. Trace how the writer is called and verify that the completed change exposes and forwards num_proc so SQL conversion can use the requested process count.
Written by the indexing model from the issue text.
Description
The underlying SqlDatasetWriter has num_proc as an available argument here , but Dataset.to_sql() does not accept it, therefore it is always using one process for the SQL conversion.
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 15
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from huggingface/datasets
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/datasets#8618 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/datasets#8617 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
huggingface/datasets#8608 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
huggingface/datasets#8590 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
huggingface/datasets#8543 · 2 comments ·
All issues in huggingface/datasets
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100