`.sel()` fails on `datetime64[s]` object

Aperta
#10,975 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
numpy, pandas, python
Ambito
data

Direzione di ricerca

Inizia con l’MVCE NumPy/xarray fornito e traccia il percorso della coordinata temporale utilizzato da DataArray.sel per gli slice di stringhe e i valori datetime64[s]. Conferma il fallimento rispetto al traceback pandas mostrato; il lavoro è completato quando la selezione 2400–2420 riesce senza richiedere cftime ed è coperta da un test di regressione.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug upstream issue
What happened?

Hi there,

sorry if this might be a duplicate, but I have been browsing the repo without finding anything specific which resemble this.

So, I am exploring to the possibility of calling xarray with CFDatetimeDecoder on time period overshooting pandas threshold year 2262
Running with xarray=2025.9.0

import xarray as xr
files="/ec/res4/scratch/ecme3497/ece4/pic2/output/oifs/pic2_atm_cmip6_1m_24*.nc"
coder = xr.coders.CFDatetimeCoder(time_unit='s')
data = xr.open_mfdataset(files, decode_times=coder)
data.time_counter

which gives me

array(['2400-01-16T12:00:00', '2400-02-15T12:00:00', '2400-03-16T12:00:00',
       ..., '2489-10-16T12:00:00', '2489-11-16T00:00:00',
       '2489-12-16T12:00:00'], shape=(1080,), dtype='datetime64[s]')

so far so good. However

data.sel(time_counter=slice("2400-01-01", "2420-01-01"))
OverflowError                             Traceback (most recent call last)
File pandas/_libs/tslibs/period.pyx:1169, in pandas._libs.tslibs.period.period_ordinal_to_dt64()

OverflowError: Overflow occurred in npy_datetimestruct_to_datetime

The above exception was the direct cause of the following exception:

OutOfBoundsDatetime                       Traceback (most recent call last)
Cell In[8], [line 1](vscode-notebook-cell:?execution_count=8&line=1)
----> [1](vscode-notebook-cell:?execution_count=8&line=1) data.sel(time_counter=slice("2400-01-01", "2420-01-01"))

File /ECMWF_kQYjfeo/miniforge/envs/env1/lib/python3.12/site-packages/xarray/core/dataset.py:2974, in Dataset.sel(self, indexers, method, tolerance, drop, **indexers_kwargs)
   2906 """Returns a new dataset with each array indexed by tick labels
   2907 along the specified dimension(s).
   2908 
   (...)   2971 
   2972 """
   2973 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "sel")
-> [2974](https://vscode-remote+ssh-002dremote-002bhpc-002dlogin.vscode-resource.vscode-cdn.net/ECMWF_kQYjfeo/miniforge/envs/env1/lib/python3.12/site-packages/xarray/core/dataset.py:2974) query_results = map_index_queries(
   2975     self, indexers=indexers, method=method, tolerance=tolerance
   2976 )
   2978 if drop:
   2979     no_scalar_variables = {}
...
File pandas/_libs/tslibs/period.pyx:1992, in pandas._libs.tslibs.period._Period.to_timestamp()

File pandas/_libs/tslibs/period.pyx:1172, in pandas._libs.tslibs.period.period_ordinal_to_dt64()

OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 2400-01-01 00:00:00

I know this can be fixed using cftime, but I though that using datetime64[s] would have fix this issue. In general, I am quite confused on what are the currently supported features of xarray for going beyond the the nanoseconds limitation of pandas. Shall we keep working with cftime? Is there something I am missing? To what extent can we used the Coder if this is the output?
Any help, also suggestion to documentation, is greatly appreciated!
Thanks a lot

EDIT: I can upload some of the data required, but below you can find a reproducible example

What did you expect to happen?

As expected, correct time selection.

Minimal Complete Verifiable Example
import numpy as np
import xarray as xr

# monthly starts from 2389-01 to 2489-12 (inclusi)
months = np.arange("2389-01", "2490-01", dtype="datetime64[M]")  # dtype M = month starts
times = months.astype("datetime64[s]")  # convert to seconds (YYYY-MM-01T00:00:00)
# creare DataArray di esempio
da = xr.DataArray(np.zeros(times.size), coords={"time": times}, dims=["time"])
da.sel(time=slice("2400-01-01", "2420-01-01"))

I have been able to replicate this also in most recent version of xarray

>>> xr.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.14.0 | packaged by conda-forge | (main, Dec  2 2025, 20:23:19) [Clang 20.1.8 ]
python-bits: 64
OS: Darwin
OS-release: 23.6.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: ('it_IT', 'UTF-8')
libhdf5: None
libnetcdf: None

xarray: 2025.11.0
pandas: 2.3.3
numpy: 2.3.5
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: None
pip: 25.3
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None
MVCE confirmation
  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Environment

xarray>=2025.9.0

Lingua principale
Python
Stelle
4.2k
Fork
1.5k
Merge medio
2g 20h
PR unite (30g)
70

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pydata/xarray

Tutte le issue di pydata/xarray

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.