Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

`TIMESTAMP`, `TIMESTAMP_NTZ`, `TINYINT` and the other `TypeDecorator`s don't declare `python_type`

Aberta Para iniciantes
  • Algo errado?
#81 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
2/5
Tempo estimado
1-3 horas
Facilidade para iniciantes
78/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
python, sqlalchemy
Domínio
backend, database

Direção de pesquisa

Comece em src/databricks/sqlalchemy/_types.py, onde as cinco subclasses de TypeDecorator estão definidas, e leia suas implementações e os testes de tipos existentes. Verifique se o python_type de cada tipo corresponde aos valores que ele retorna: datetime para TIMESTAMP e TIMESTAMP_NTZ, int para TINYINT, time para DatabricksTimeType e str para DatabricksStringType. O trabalho estará concluído quando esses tipos indicarem as classes Python esperadas nas versões compatíveis do SQLAlchemy.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Description of the bug
Our use case

We generate SQLAlchemy ORM models from Databricks tables with sqlacodegen. It reflects the tables through this dialect and annotates each column with the column type's python_type, for example Mapped[str | None] for a STRING column.

For TIMESTAMP, TIMESTAMP_NTZ and TINYINT columns we get Mapped[object | None] instead of Mapped[datetime | None] and Mapped[int | None]. These are the types reflection returns for timestamp, timestamp_ntz and tinyint. Type checkers then accept any value for these attributes, and any code reading them sees object.

The same happens to any other tool that reads python_type, and to the two types the dialect uses in place of Time and String through its colspecs (DatabricksTimeType and DatabricksStringType).

The bug

These five types are sqlalchemy.types.TypeDecorator subclasses that don't declare python_type:

Type impl Values returned python_type
TIMESTAMP DateTime datetime object
TIMESTAMP_NTZ DateTime datetime object
TINYINT SmallInteger int object
DatabricksTimeType Time time object
DatabricksStringType String str object

TypeDecorator doesn't take python_type from its impl, so these types get the TypeEngine default: object in SQLAlchemy 2.1, and a NotImplementedError in 2.0.

The SQLAlchemy maintainers consider this the intended behaviour. A TypeDecorator can return different values from its impl through process_result_value, so only the decorator knows its Python type, and it's expected to declare python_type itself: link to the SQLAlchemy discussion.

Versions
  • databricks-sqlalchemy: 2.0.10 (also on main)
  • databricks-sql-connector: 4.6.0
  • SQLAlchemy: 2.1.4, and 2.0.54
  • Python: 3.12
  • OS: Linux
To Reproduce

No connection is needed:

import datetime
from importlib.metadata import version

import sqlalchemy as sa
from databricks.sqlalchemy import TIMESTAMP, TIMESTAMP_NTZ, TINYINT, DatabricksDialect

print("SQLAlchemy", sa.__version__)
print("databricks-sqlalchemy", version("databricks-sqlalchemy"))

dialect = DatabricksDialect()
expected = {
    # The types reflection returns for timestamp, timestamp_ntz and tinyint.
    "TIMESTAMP": (TIMESTAMP(), datetime.datetime),
    "TIMESTAMP_NTZ": (TIMESTAMP_NTZ(), datetime.datetime),
    "TINYINT": (TINYINT(), int),
    # The types the dialect uses in place of Time and String.
    "Time": (sa.Time().dialect_impl(dialect), datetime.time),
    "String": (sa.String().dialect_impl(dialect), str),
}
for name, (type_, python_type) in expected.items():
    print(f"{name:14} {type(type_).__name__:20} python_type: {type_.python_type}")

for name, (type_, python_type) in expected.items():
    assert type_.python_type is python_type, name
Output

With SQLAlchemy 2.1.4:

SQLAlchemy 2.1.4
databricks-sqlalchemy 2.0.10
TIMESTAMP      TIMESTAMP            python_type: <class 'object'>
TIMESTAMP_NTZ  TIMESTAMP_NTZ        python_type: <class 'object'>
TINYINT        TINYINT              python_type: <class 'object'>
Time           DatabricksTimeType   python_type: <class 'object'>
String         DatabricksStringType python_type: <class 'object'>
Traceback (most recent call last):
  ...
AssertionError: TIMESTAMP

With SQLAlchemy 2.0.54, the first python_type raises NotImplementedError.

Expected behaviour

Each type's python_type is the type of the values it returns: datetime for TIMESTAMP and TIMESTAMP_NTZ, int for TINYINT, time for DatabricksTimeType and str for DatabricksStringType.

Possible fix

Declare python_type on each type in src/databricks/sqlalchemy/_types.py, which already imports datetime and time from datetime:

class TIMESTAMP_NTZ(sqlalchemy.types.TypeDecorator):
    ...

    @property
    def python_type(self) -> type:
        return datetime

The same goes for TIMESTAMP (datetime), TINYINT (int), DatabricksTimeType (time) and DatabricksStringType (str).

Additional context

  • DatabricksArray, DatabricksMap and DatabricksVariant also report object, because UserDefinedType has the same default. That's right for all three today. VARIANT can hold any value, and ARRAY and MAP have no result_processor, so their values are whatever the connector returns, which depends on its settings:

    databricks-sql-connector 4.6.0 ARRAY MAP
    default numpy.ndarray list of (key, value) tuples
    disable_pandas=True list list of (key, value) tuples
    _use_arrow_native_complex_types=False str (JSON) str (JSON)

    A result_processor converting the values to list and dict would let these two types declare python_type too, but that changes the values users get, so it's not part of this request.

  • Our workaround: when a TypeDecorator reports object, we use its impl's python_type. That's correct for the five types above, but as the SQLAlchemy maintainers point out, it's only a guess in general.

Thank you!

Linguagem predominante
Python
Estrelas
24
Forks
20
Nenhum PR com merge em 30d

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de databricks/databricks-sqlalchemy

Todas as issues de databricks/databricks-sqlalchemy

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.