Support Python 3.15 and free-threaded 3.15t

未关闭
#11,710 6 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
28/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
numpy, pandas, python
领域
backend, ci-cd, testing

调研方向

Start by reading the Python test environment definitions and CI configuration for the existing 3.14 environments, then review the named failing tests in xarray/testing/assertions.py and the test guards described in the report. Compare the proposed 3.15 and 3.15t environments with those setups and investigate the scipy file manager race. Done means the new environments and fixes are ready for CI, with the reported failures addressed.

由索引模型根据 Issue 内容生成。

描述

Python 3.15.0 is out, and conda-forge has both python-gil and python-freethreading builds. I tried running the test suite on both, as preparation for adding them to CI.

Python 3.15 (conda-forge)
  • All test dependencies have 3.15 builds except pyarrow.
  • The dask metapackage requires pyarrow, so without a lower bound the solver silently falls back to dask/distributed 2023.3.0. Using dask-core >=2026 + distributed instead gives dask 2026.8.0, and otherwise the same versions as test-py314.
  • Test results: 1 failed, 22016 passed. TestDataArray::test_to_dask_dataframe needs pyarrow (via dask.dataframe) but is only guarded by requires_dask.
Free-threaded Python 3.15t (PyPI wheels)

conda-forge has no free-threaded 3.15 builds of numpy, pandas, scipy etc. yet, but PyPI has cp315t wheels for numpy, pandas, scipy, pyarrow, matplotlib, cartopy, rasterio and lxml. There are no cp315t wheels for cftime, netCDF4, h5py, numcodecs, bottleneck, numexpr and numba.

With a conda python-freethreading interpreter and PyPI wheels: 150 failed, 2 errors, 14646 passed. With PYTHON_GIL=1, 148 of these still fail, so most aren't free-threading issues:

  • 137: h5netcdf is installed without h5py (no cp315t wheel), so requires_h5netcdf doesn't skip. Environment issue.
  • 5: test_ensure_warnings_not_elevated. ensure_warnings in xarray/testing/assertions.py reassigns warnings.filters, which has no effect with context-aware warnings (sys.flags.context_aware_warnings, on by default in free-threaded builds). So the public xr.testing.assert_* functions elevate warnings to errors there.
  • 3: test_rechunk_multi_dimensional_da_cftime is missing requires_cftime.
  • 3: datatree tests assume netCDF4 is installed (NameError: name 'nc4' is not defined in a fixture, and test_default_write_engine).

Only 4 failures depend on the GIL: test_open_mfdataset_manyfiles[scipy-20-*-*-5] (file_cache_maxsize=5, parallel dask reads). CachingFileManager evicts and closes a scipy file opened with mmap while another thread still uses its data. The resulting RuntimeWarning leaves the LRU cache inconsistent and ends in a KeyError. These pass with PYTHON_GIL=1.

Upstream status

Python 3.15 (checked when there are builds on conda-forge and wheels on PyPI):

  • numpy: 2.5.4 on conda-forge and PyPI
  • numba: 0.68.0 on conda-forge and PyPI (numba/numba#10581)
  • dask: pure Python (on conda-forge use dask-core; the dask metapackage needs pyarrow)
  • netCDF4: abi3 builds of 1.7.4 on conda-forge and PyPI
  • h5netcdf: pure Python
  • bottleneck: 1.6.0 on conda-forge; PyPI wheels need a release (support merged in pydata/bottleneck#589, release requested in pydata/bottleneck#591)
    Check when: a bottleneck release after 1.6.0 ships cp315 wheels on PyPI.
  • cftime: 1.6.5 on conda-forge, abi3 wheels of 1.6.6 on PyPI
  • numcodecs: 0.17.0 on conda-forge; no PyPI wheels: zarr-developers/numcodecs#874 (zarr-developers/numcodecs#842 was closed without merging)
    Check when: a numcodecs release ships cp315 wheels on PyPI (0.17.0 doesn't).
  • pyarrow: wheels of 26.0.0 on PyPI; py315 builds of 25.0.0 on conda-forge (conda-forge/pyarrow-feedstock#184)
  • h5py: 3.16.0 on conda-forge; no PyPI wheels: h5py/h5py#2972 (closed; abi3 wheels from h5py/h5py#2885 are not released yet)
    Check when: an h5py release ships cp315 or abi3 wheels on PyPI (3.16.0 has neither).
  • numexpr: 2.14.2 on conda-forge; no PyPI wheels: pydata/numexpr#579
    Check when: a numexpr release ships cp315 wheels on PyPI (2.14.2 doesn't).
  • cf-units (needed by iris): abi3 builds of 3.3.1 on conda-forge and PyPI
  • pyyaml (needed by dask): 6.0.3 on conda-forge; no PyPI wheels: yaml/pyyaml#951
    Check when: a pyyaml release ships cp315 wheels on PyPI (6.0.3 doesn't).

Free-threaded Python 3.15t (checked when there are cp315t wheels on PyPI):

  • conda-forge builds: the py315 migration started in conda-forge/conda-forge-pinning-feedstock#8911, but the free-threading migration only covers 3.14t so far
    Check when: conda-forge starts a python315t migration and numpy/pandas/scipy have cp315t builds.
  • numpy: cp315t wheels of 2.5.4
  • numba (also needed by numbagg): numba/numba#10473 (free-threading in general); 3.15t support is announced for the next release in numba/numba#10581
    Check when: a numba release ships cp315t wheels on PyPI (0.68.0 only has cp315).
  • dask: pure Python
  • netCDF4: cp315t wheels of 1.7.5 (Unidata/netcdf4-python#1509)
  • h5netcdf: pure Python (but needs h5py, see below)
  • bottleneck: release requested in pydata/bottleneck#591 (see also pydata/bottleneck#532 on free-threading safety)
    Check when: a bottleneck release ships cp315t wheels on PyPI.
  • cftime (also needed by nc-time-axis and iris): cp315t wheels of 1.6.6.1 (Unidata/cftime#421, Unidata/cftime#424)
  • numcodecs (also needed by zarr): zarr-developers/numcodecs#874 (see also zarr-developers/numcodecs#749 for free-threading in general; 3.14t support in zarr-developers/numcodecs#858)
    Check when: a numcodecs release ships cp315t wheels on PyPI (0.17.0 doesn't).
  • pyarrow: cp315t wheels of 26.0.0
  • h5py: h5py/h5py#2475 (free-threading support; abi3t wheels planned, see h5py/h5py#2972)
    Check when: an h5py release ships cp315t or abi3t wheels on PyPI.
  • numexpr: pydata/numexpr#579 (2.14.2 already has cp314t wheels, so the next release will likely include cp315t)
    Check when: a numexpr release ships cp315t wheels on PyPI.
  • cf-units (needed by iris): no issue yet; 3.3.1 only has abi3 wheels
    Check when: a cf-units release ships cp315t or abi3t wheels on PyPI.
  • pyyaml (needed by dask): yaml/pyyaml#951 (6.0.3 already has cp314t wheels; see also yaml/pyyaml#870 for free-threading in general)
    Check when: a pyyaml release ships cp315t wheels on PyPI.

Other test dependencies without 3.15t wheels on PyPI (checked 2026-10-13): psutil and tornado (distributed), google-crc32c (zarr) and lz4 (dask extras). psutil, tornado and lz4 build from source on 3.15t; google-crc32c builds but falls back to a pure Python implementation with a RuntimeWarning on import. cf-units doesn't build without udunits2, so iris has to be left out of the 3.15t env.

Proposal for CI
  • Add test-py315 alongside the 3.14 envs, and only replace 3.14 once pyarrow has 3.15 builds on conda-forge. Until then, move pyarrow from the accel feature into its own feature and use dask-core in the 3.15 env.
  • Add test-py315t (ubuntu only) with a conda python-freethreading interpreter and PyPI wheels, without h5netcdf until h5py has cp315t wheels.
  • Fix the test guards and ensure_warnings above, and look into the file manager race.

[This is Claude Code on behalf of Michael Niklas]

主要语言
Python
星标
4.2k
派生
1.5k
平均合并
2 天 21 小时
30 天内合并 PR
74

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

pydata/xarray 的其他 Issue

查看 pydata/xarray 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。