Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

`TIMESTAMP`, `TIMESTAMP_NTZ`, `TINYINT` and the other `TypeDecorator`s don't declare `python_type`

Open Beginner friendly
  • Is something wrong?
#81 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, sqlalchemy
Domain
backend, database

Research direction

Start in src/databricks/sqlalchemy/_types.py, where the five TypeDecorator subclasses are defined, and read their implementations and existing type tests. Check each type’s python_type against the values it returns: datetime for TIMESTAMP and TIMESTAMP_NTZ, int for TINYINT, time for DatabricksTimeType, and str for DatabricksStringType. Done when those types report the expected Python classes under the supported SQLAlchemy versions.

Written by the indexing model from the issue text.

Description

Description of the bug
Our use case

We generate SQLAlchemy ORM models from Databricks tables with sqlacodegen. It reflects the tables through this dialect and annotates each column with the column type's python_type, for example Mapped[str | None] for a STRING column.

For TIMESTAMP, TIMESTAMP_NTZ and TINYINT columns we get Mapped[object | None] instead of Mapped[datetime | None] and Mapped[int | None]. These are the types reflection returns for timestamp, timestamp_ntz and tinyint. Type checkers then accept any value for these attributes, and any code reading them sees object.

The same happens to any other tool that reads python_type, and to the two types the dialect uses in place of Time and String through its colspecs (DatabricksTimeType and DatabricksStringType).

The bug

These five types are sqlalchemy.types.TypeDecorator subclasses that don't declare python_type:

Type impl Values returned python_type
TIMESTAMP DateTime datetime object
TIMESTAMP_NTZ DateTime datetime object
TINYINT SmallInteger int object
DatabricksTimeType Time time object
DatabricksStringType String str object

TypeDecorator doesn't take python_type from its impl, so these types get the TypeEngine default: object in SQLAlchemy 2.1, and a NotImplementedError in 2.0.

The SQLAlchemy maintainers consider this the intended behaviour. A TypeDecorator can return different values from its impl through process_result_value, so only the decorator knows its Python type, and it's expected to declare python_type itself: link to the SQLAlchemy discussion.

Versions
  • databricks-sqlalchemy: 2.0.10 (also on main)
  • databricks-sql-connector: 4.6.0
  • SQLAlchemy: 2.1.4, and 2.0.54
  • Python: 3.12
  • OS: Linux
To Reproduce

No connection is needed:

import datetime
from importlib.metadata import version

import sqlalchemy as sa
from databricks.sqlalchemy import TIMESTAMP, TIMESTAMP_NTZ, TINYINT, DatabricksDialect

print("SQLAlchemy", sa.__version__)
print("databricks-sqlalchemy", version("databricks-sqlalchemy"))

dialect = DatabricksDialect()
expected = {
    # The types reflection returns for timestamp, timestamp_ntz and tinyint.
    "TIMESTAMP": (TIMESTAMP(), datetime.datetime),
    "TIMESTAMP_NTZ": (TIMESTAMP_NTZ(), datetime.datetime),
    "TINYINT": (TINYINT(), int),
    # The types the dialect uses in place of Time and String.
    "Time": (sa.Time().dialect_impl(dialect), datetime.time),
    "String": (sa.String().dialect_impl(dialect), str),
}
for name, (type_, python_type) in expected.items():
    print(f"{name:14} {type(type_).__name__:20} python_type: {type_.python_type}")

for name, (type_, python_type) in expected.items():
    assert type_.python_type is python_type, name
Output

With SQLAlchemy 2.1.4:

SQLAlchemy 2.1.4
databricks-sqlalchemy 2.0.10
TIMESTAMP      TIMESTAMP            python_type: <class 'object'>
TIMESTAMP_NTZ  TIMESTAMP_NTZ        python_type: <class 'object'>
TINYINT        TINYINT              python_type: <class 'object'>
Time           DatabricksTimeType   python_type: <class 'object'>
String         DatabricksStringType python_type: <class 'object'>
Traceback (most recent call last):
  ...
AssertionError: TIMESTAMP

With SQLAlchemy 2.0.54, the first python_type raises NotImplementedError.

Expected behaviour

Each type's python_type is the type of the values it returns: datetime for TIMESTAMP and TIMESTAMP_NTZ, int for TINYINT, time for DatabricksTimeType and str for DatabricksStringType.

Possible fix

Declare python_type on each type in src/databricks/sqlalchemy/_types.py, which already imports datetime and time from datetime:

class TIMESTAMP_NTZ(sqlalchemy.types.TypeDecorator):
    ...

    @property
    def python_type(self) -> type:
        return datetime

The same goes for TIMESTAMP (datetime), TINYINT (int), DatabricksTimeType (time) and DatabricksStringType (str).

Additional context

  • DatabricksArray, DatabricksMap and DatabricksVariant also report object, because UserDefinedType has the same default. That's right for all three today. VARIANT can hold any value, and ARRAY and MAP have no result_processor, so their values are whatever the connector returns, which depends on its settings:

    databricks-sql-connector 4.6.0 ARRAY MAP
    default numpy.ndarray list of (key, value) tuples
    disable_pandas=True list list of (key, value) tuples
    _use_arrow_native_complex_types=False str (JSON) str (JSON)

    A result_processor converting the values to list and dict would let these two types declare python_type too, but that changes the values users get, so it's not part of this request.

  • Our workaround: when a TypeDecorator reports object, we use its impl's python_type. That's correct for the five types above, but as the SQLAlchemy maintainers point out, it's only a guess in general.

Thank you!

Dominant language
Python
Stars
24
Forks
20
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/databricks-sqlalchemy

All issues in databricks/databricks-sqlalchemy

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.