Lift the Python <3.15 cap once dill supports Python 3.15
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 83/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- backend, build-system
Research direction
Read CondaPkg.toml and the matching notes in AGENTS.md; first confirm that a dill release with uqfoundation/dill#755 is available on conda-forge. Remove the Python version cap and add a dill lower bound if needed. Run the num_proc test in test/dataset.jl and the docs build; done when fresh environments can resolve Python 3.15 and both checks pass.
Written by the indexing model from the issue text.
Description
#95 caps Python at <3.15 in CondaPkg.toml, so new environments get 3.14. Lift the cap once dill releases support for 3.15.
On Python 3.15, dill 0.4.1 (the latest release on PyPI and conda-forge) can't pickle functions, because 3.15 removed code.co_lnotab:
>>> dill.dumps(lambda b: b)
AttributeError: 'code' object has no attribute 'co_lnotab'
datasets depends on dill in two places, which both fail on 3.15:
num_procsends the function to its worker processes with dill.ds.py.map(pyfn; num_proc=2)raises theAttributeErrorabove; this is what failed on CI in #85.map/filterfingerprint the transform with dill. On 3.15 this falls back to "couldn't be hashed properly, a random hash was used instead", so caching never reuses a result.
The fix is merged in dill (uqfoundation/dill#755, for uqfoundation/dill#753) but not released yet.
To lift the cap: once a dill release with #755 is on conda-forge, remove [deps.python] version = "<3.15" from CondaPkg.toml. If needed, add a dill lower bound. Also update the matching notes in AGENTS.md. CI's fresh environments then resolve 3.15 again. Check that the num_proc test in test/dataset.jl and the docs build pass.
- Dominant language
- Julia
- Stars
- 38
- Forks
- 1
- Avg merge
- 4h 1m
- Merged PRs (30d)
- 16
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from JuliaGenAI/HuggingFaceDatasets.jl
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
JuliaGenAI/HuggingFaceDatasets.jl#48 ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
JuliaGenAI/HuggingFaceDatasets.jl#32 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
JuliaGenAI/HuggingFaceDatasets.jl#29 · 1 comment ·
Maintainers usually reply within 1 day
All issues in JuliaGenAI/HuggingFaceDatasets.jl
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
JuliaData/DBFTables.jl#50 ·
-
code-review
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
JuliaNLSolvers/Optim.jl#1296 ·
-
OptimizationCUTEst jobs fail: gfortran not installed on the runnerPossibly taken @ChrisRackauckas-Claude claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
SciML/SciMLBenchmarks.jl#1946 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
JuliaSmoothOptimizers/NCL.jl#101 ·
Maintainers usually reply within 1 day