Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

`TIMESTAMP`, `TIMESTAMP_NTZ`, `TINYINT` and the other `TypeDecorator`s don't declare `python_type`

Ouverte Adaptée aux débutants
  • Quelque chose ne va pas ?
#81 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
2/5
Temps estimé
1-3 heures
Accessibilité débutants
78/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
python, sqlalchemy
Domaine
backend, database

Piste de recherche

Commencez dans src/databricks/sqlalchemy/_types.py, où sont définies les cinq sous-classes de TypeDecorator, et lisez leurs implémentations ainsi que les tests de types existants. Vérifiez que le python_type de chaque type corresponde aux valeurs qu’il renvoie : datetime pour TIMESTAMP et TIMESTAMP_NTZ, int pour TINYINT, time pour DatabricksTimeType et str pour DatabricksStringType. Le travail est terminé lorsque ces types indiquent les classes Python attendues avec les versions de SQLAlchemy prises en charge.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Description of the bug
Our use case

We generate SQLAlchemy ORM models from Databricks tables with sqlacodegen. It reflects the tables through this dialect and annotates each column with the column type's python_type, for example Mapped[str | None] for a STRING column.

For TIMESTAMP, TIMESTAMP_NTZ and TINYINT columns we get Mapped[object | None] instead of Mapped[datetime | None] and Mapped[int | None]. These are the types reflection returns for timestamp, timestamp_ntz and tinyint. Type checkers then accept any value for these attributes, and any code reading them sees object.

The same happens to any other tool that reads python_type, and to the two types the dialect uses in place of Time and String through its colspecs (DatabricksTimeType and DatabricksStringType).

The bug

These five types are sqlalchemy.types.TypeDecorator subclasses that don't declare python_type:

Type impl Values returned python_type
TIMESTAMP DateTime datetime object
TIMESTAMP_NTZ DateTime datetime object
TINYINT SmallInteger int object
DatabricksTimeType Time time object
DatabricksStringType String str object

TypeDecorator doesn't take python_type from its impl, so these types get the TypeEngine default: object in SQLAlchemy 2.1, and a NotImplementedError in 2.0.

The SQLAlchemy maintainers consider this the intended behaviour. A TypeDecorator can return different values from its impl through process_result_value, so only the decorator knows its Python type, and it's expected to declare python_type itself: link to the SQLAlchemy discussion.

Versions
  • databricks-sqlalchemy: 2.0.10 (also on main)
  • databricks-sql-connector: 4.6.0
  • SQLAlchemy: 2.1.4, and 2.0.54
  • Python: 3.12
  • OS: Linux
To Reproduce

No connection is needed:

import datetime
from importlib.metadata import version

import sqlalchemy as sa
from databricks.sqlalchemy import TIMESTAMP, TIMESTAMP_NTZ, TINYINT, DatabricksDialect

print("SQLAlchemy", sa.__version__)
print("databricks-sqlalchemy", version("databricks-sqlalchemy"))

dialect = DatabricksDialect()
expected = {
    # The types reflection returns for timestamp, timestamp_ntz and tinyint.
    "TIMESTAMP": (TIMESTAMP(), datetime.datetime),
    "TIMESTAMP_NTZ": (TIMESTAMP_NTZ(), datetime.datetime),
    "TINYINT": (TINYINT(), int),
    # The types the dialect uses in place of Time and String.
    "Time": (sa.Time().dialect_impl(dialect), datetime.time),
    "String": (sa.String().dialect_impl(dialect), str),
}
for name, (type_, python_type) in expected.items():
    print(f"{name:14} {type(type_).__name__:20} python_type: {type_.python_type}")

for name, (type_, python_type) in expected.items():
    assert type_.python_type is python_type, name
Output

With SQLAlchemy 2.1.4:

SQLAlchemy 2.1.4
databricks-sqlalchemy 2.0.10
TIMESTAMP      TIMESTAMP            python_type: <class 'object'>
TIMESTAMP_NTZ  TIMESTAMP_NTZ        python_type: <class 'object'>
TINYINT        TINYINT              python_type: <class 'object'>
Time           DatabricksTimeType   python_type: <class 'object'>
String         DatabricksStringType python_type: <class 'object'>
Traceback (most recent call last):
  ...
AssertionError: TIMESTAMP

With SQLAlchemy 2.0.54, the first python_type raises NotImplementedError.

Expected behaviour

Each type's python_type is the type of the values it returns: datetime for TIMESTAMP and TIMESTAMP_NTZ, int for TINYINT, time for DatabricksTimeType and str for DatabricksStringType.

Possible fix

Declare python_type on each type in src/databricks/sqlalchemy/_types.py, which already imports datetime and time from datetime:

class TIMESTAMP_NTZ(sqlalchemy.types.TypeDecorator):
    ...

    @property
    def python_type(self) -> type:
        return datetime

The same goes for TIMESTAMP (datetime), TINYINT (int), DatabricksTimeType (time) and DatabricksStringType (str).

Additional context

  • DatabricksArray, DatabricksMap and DatabricksVariant also report object, because UserDefinedType has the same default. That's right for all three today. VARIANT can hold any value, and ARRAY and MAP have no result_processor, so their values are whatever the connector returns, which depends on its settings:

    databricks-sql-connector 4.6.0 ARRAY MAP
    default numpy.ndarray list of (key, value) tuples
    disable_pandas=True list list of (key, value) tuples
    _use_arrow_native_complex_types=False str (JSON) str (JSON)

    A result_processor converting the values to list and dict would let these two types declare python_type too, but that changes the values users get, so it's not part of this request.

  • Our workaround: when a TypeDecorator reports object, we use its impl's python_type. That's correct for the five types above, but as the SQLAlchemy maintainers point out, it's only a guess in general.

Thank you!

Langage dominant
Python
Étoiles
24
Forks
20
Aucune PR mergée en 30 j

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de databricks/databricks-sqlalchemy

Toutes les issues de databricks/databricks-sqlalchemy

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.