Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

`TIMESTAMP`, `TIMESTAMP_NTZ`, `TINYINT` and the other `TypeDecorator`s don't declare `python_type`

Abierto Apto para principiantes
  • ¿Algo no está bien?
#81 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
78/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
python, sqlalchemy
Área
backend, database

Línea de trabajo

Empieza en src/databricks/sqlalchemy/_types.py, donde se definen las cinco subclases de TypeDecorator, y lee sus implementaciones y las pruebas de tipos existentes. Comprueba que el python_type de cada tipo coincida con los valores que devuelve: datetime para TIMESTAMP y TIMESTAMP_NTZ, int para TINYINT, time para DatabricksTimeType y str para DatabricksStringType. La tarea estará terminada cuando esos tipos indiquen las clases de Python esperadas en las versiones de SQLAlchemy compatibles.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Description of the bug
Our use case

We generate SQLAlchemy ORM models from Databricks tables with sqlacodegen. It reflects the tables through this dialect and annotates each column with the column type's python_type, for example Mapped[str | None] for a STRING column.

For TIMESTAMP, TIMESTAMP_NTZ and TINYINT columns we get Mapped[object | None] instead of Mapped[datetime | None] and Mapped[int | None]. These are the types reflection returns for timestamp, timestamp_ntz and tinyint. Type checkers then accept any value for these attributes, and any code reading them sees object.

The same happens to any other tool that reads python_type, and to the two types the dialect uses in place of Time and String through its colspecs (DatabricksTimeType and DatabricksStringType).

The bug

These five types are sqlalchemy.types.TypeDecorator subclasses that don't declare python_type:

Type impl Values returned python_type
TIMESTAMP DateTime datetime object
TIMESTAMP_NTZ DateTime datetime object
TINYINT SmallInteger int object
DatabricksTimeType Time time object
DatabricksStringType String str object

TypeDecorator doesn't take python_type from its impl, so these types get the TypeEngine default: object in SQLAlchemy 2.1, and a NotImplementedError in 2.0.

The SQLAlchemy maintainers consider this the intended behaviour. A TypeDecorator can return different values from its impl through process_result_value, so only the decorator knows its Python type, and it's expected to declare python_type itself: link to the SQLAlchemy discussion.

Versions
  • databricks-sqlalchemy: 2.0.10 (also on main)
  • databricks-sql-connector: 4.6.0
  • SQLAlchemy: 2.1.4, and 2.0.54
  • Python: 3.12
  • OS: Linux
To Reproduce

No connection is needed:

import datetime
from importlib.metadata import version

import sqlalchemy as sa
from databricks.sqlalchemy import TIMESTAMP, TIMESTAMP_NTZ, TINYINT, DatabricksDialect

print("SQLAlchemy", sa.__version__)
print("databricks-sqlalchemy", version("databricks-sqlalchemy"))

dialect = DatabricksDialect()
expected = {
    # The types reflection returns for timestamp, timestamp_ntz and tinyint.
    "TIMESTAMP": (TIMESTAMP(), datetime.datetime),
    "TIMESTAMP_NTZ": (TIMESTAMP_NTZ(), datetime.datetime),
    "TINYINT": (TINYINT(), int),
    # The types the dialect uses in place of Time and String.
    "Time": (sa.Time().dialect_impl(dialect), datetime.time),
    "String": (sa.String().dialect_impl(dialect), str),
}
for name, (type_, python_type) in expected.items():
    print(f"{name:14} {type(type_).__name__:20} python_type: {type_.python_type}")

for name, (type_, python_type) in expected.items():
    assert type_.python_type is python_type, name
Output

With SQLAlchemy 2.1.4:

SQLAlchemy 2.1.4
databricks-sqlalchemy 2.0.10
TIMESTAMP      TIMESTAMP            python_type: <class 'object'>
TIMESTAMP_NTZ  TIMESTAMP_NTZ        python_type: <class 'object'>
TINYINT        TINYINT              python_type: <class 'object'>
Time           DatabricksTimeType   python_type: <class 'object'>
String         DatabricksStringType python_type: <class 'object'>
Traceback (most recent call last):
  ...
AssertionError: TIMESTAMP

With SQLAlchemy 2.0.54, the first python_type raises NotImplementedError.

Expected behaviour

Each type's python_type is the type of the values it returns: datetime for TIMESTAMP and TIMESTAMP_NTZ, int for TINYINT, time for DatabricksTimeType and str for DatabricksStringType.

Possible fix

Declare python_type on each type in src/databricks/sqlalchemy/_types.py, which already imports datetime and time from datetime:

class TIMESTAMP_NTZ(sqlalchemy.types.TypeDecorator):
    ...

    @property
    def python_type(self) -> type:
        return datetime

The same goes for TIMESTAMP (datetime), TINYINT (int), DatabricksTimeType (time) and DatabricksStringType (str).

Additional context

  • DatabricksArray, DatabricksMap and DatabricksVariant also report object, because UserDefinedType has the same default. That's right for all three today. VARIANT can hold any value, and ARRAY and MAP have no result_processor, so their values are whatever the connector returns, which depends on its settings:

    databricks-sql-connector 4.6.0 ARRAY MAP
    default numpy.ndarray list of (key, value) tuples
    disable_pandas=True list list of (key, value) tuples
    _use_arrow_native_complex_types=False str (JSON) str (JSON)

    A result_processor converting the values to list and dict would let these two types declare python_type too, but that changes the values users get, so it's not part of this request.

  • Our workaround: when a TypeDecorator reports object, we use its impl's python_type. That's correct for the five types above, but as the SQLAlchemy maintainers point out, it's only a guess in general.

Thank you!

Lenguaje dominante
Python
Estrellas
24
Forks
20
Sin PR fusionados en 30 d

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de databricks/databricks-sqlalchemy

Todos los issues de databricks/databricks-sqlalchemy

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.