[db-dtypes] Support pandas 3.1 ExtensionArray compliance interface changes

Aperta
#18,514 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
numpy, pandas, python

Direzione di ricerca

Start with db_dtypes/dtypes.py and the compliance files tests/compliance/test_date_compliance.py, test_json_compliance.py, and test_time_compliance.py. Run nox -s unit_prerelease with pandas 3.1.0rc0, then inspect the failing ExtensionArray tests and the temporary constraint in packages/db-dtypes/testing/constraints-3.14.txt. Done means unit and compliance suites pass against pandas 3.1 and the temporary ceiling is removed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

When testing with pandas >= 3.1.0 (currently in pre-release on PyPI as pandas==3.1.0rc0), the db-dtypes compliance test suite (tests/compliance) fails due to upstream changes in pandas internal utilities and ExtensionArray interface contracts.

To prevent presubmit PR CI from failing unexpectedly on developer branches, a temporary ceiling constraint (pandas < 3.1.0dev0) was added in packages/db-dtypes/testing/constraints-3.14.txt. This issue tracks the feature work required to update db-dtypes for full pandas 3.1 compatibility.


Problem Details

1. Private Import Removal (np_version_gt2)

The compliance test files (test_date_compliance.py, test_json_compliance.py, test_time_compliance.py) currently import a private helper:

from pandas.compat.numpy import np_version_gt2

In pandas 3.1, this private attribute has been removed from pandas.compat.numpy, resulting in test collection errors:

ImportError: cannot import name 'np_version_gt2' from 'pandas.compat.numpy'
2. ExtensionArray Interface Contract Changes

Pandas 3.1 introduces stricter interface requirements in pandas.tests.extension.base:

  • fillna mapping: ExtensionArray.fillna() no longer accepts dictionary or mapping values.
  • Comparison Operator Outputs: Changes to boolean comparison return types and PyArrow-backed boolean handling.
  • Datetime / ZoneInfo fixture dependencies: Test fixtures generating localized timestamps require explicit tzdata resolution on newer Python runtimes.

Desired Solution

  1. Decouple Version Checks:
    Replace from pandas.compat.numpy import np_version_gt2 in tests/compliance/ with standard version checks using packaging.version.parse(np.__version__) >= Version("2.0.0").
  2. Update Array Implementations for Pandas 3.1:
    Audit and update DateArray, TimeArray, and JSONArray in db_dtypes/dtypes.py to satisfy both pandas 3.0 and pandas 3.1 ExtensionArray contracts simultaneously.
  3. Verify Pre-release & Compliance:
    Run both unit and compliance test suites against pandas>=3.1.0rc0 using nox -s unit_prerelease to ensure 100% compliance.
  4. Remove Temporary Constraint:
    Remove pandas < 3.1.0dev0 from packages/db-dtypes/testing/constraints-3.14.txt.
Lingua principale
Python
Stelle
5.4k
Fork
1.8k
Merge medio
2g 20h
PR unite (30g)
140

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di googleapis/google-cloud-python

Tutte le issue di googleapis/google-cloud-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.