Raise the minimum fsspec to 2024.5.0 in 2.4

未關閉 適合新手
#1,687 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 6 天內回覆

還沒有人認領這個 Issue。

評估

難度
2/5
預估耗時
1-3 小時
新手友好度
68/100
Issue 類型
重構
描述清晰度
描述清楚
活躍度
活躍
技術堆疊
python

研究方向

從 pyproject.toml 開始:將 fsspec、s3fs 和 gcsfs 的下限設為 2024.5.0,並檢查 adlfs 是否需要相符的下限。閱讀 #1686,了解能讓 old-fsspec 分支被移除的本地路徑正規化,然後簡化 test_path_confinement.py 中 2024.5.0 之前的案例。當新的下限已宣告、最小相依 CI 工作通過,且路徑測試不再依舊 fsspec 行為分支時即完成。

由索引模型根據 Issue 內容生成。

描述

dependencies needs-discussion

Proposal

Raise the minimum fsspec from 2023.1.0 to 2024.5.0 in 2.4, and the storage extras with it (s3fs, gcsfs), which pin fsspec to their own version.

Why

Older fsspec resolves local paths differently from current releases, and the file-store code has to handle both:

  • Before 2024.5.0, LocalFileSystem._strip_protocol(".") keeps a trailing /.. A file store with location "" or "." rejected every path because of it, which #1686 fixes by normalizing both sides of the comparison.
  • In fsspec 2023, file: and local: without // are not read as a protocol, so the same path string names a different file than on current fsspec. The test in test_path_confinement.py had to be rewritten in #1686 to hold on both.

The minimum-dependency CI job catches these differences, but only after the fact. Every change to path handling has to be reasoned through on two fsspec behaviors. Releases from 2024.5.0 on behave the same as current fsspec in both respects.

Cost

  • Users on s3fs or gcsfs older than 2024.5.0 upgrade them along with fsspec. s3fs in turn pins aiobotocore, which pins botocore, so environments that hold an old boto stack would need to move it.
  • adlfs does not pin fsspec exactly and is not affected beyond the fsspec floor itself.

Scope

  • 2.4 only. 2.3.x keeps fsspec>=2023.1.0: a patch release should not force dependency upgrades, and #1686 keeps the 2.3 line working on old fsspec.
  • pyproject.toml: fsspec>=2024.5.0, s3fs>=2024.5.0, gcsfs>=2024.5.0; check whether adlfs needs a matching floor.
  • The minimum-dependency CI job picks up the new floor from pyproject.toml.
  • Once the floor is raised, the old-fsspec branches of the path tests can be simplified.
主要語言
Python
星號
198
分支
103
平均合併
6 天 1 小時
30 天內合併 PR
10

環境準備

  • 提供 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

datajoint/datajoint-python 的其他 Issue

查看 datajoint/datajoint-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。