Unable to insert Python lists into ARRAY<STRING> columns using pandas to_sql

Start with DatabricksArray and the pandas DataFrame.to_sql INSERT path, using the provided minimal example to reproduce the empty arrays. Trace how Python list parameters are bound for ARRAY; done means both rows retain their email elements instead of becoming empty arrays, with regression coverage for both examples.

Description

Hello,

I am trying to insert Python lists into a Databricks
ARRAY<STRING> column using pandas DataFrame.to_sql() with
the Databricks SQLAlchemy dialect.

The table is created correctly with an ARRAY<STRING> column,
but the inserted arrays are stored as empty arrays.

Minimal reproducible example

import pandas as pd
from sqlalchemy import create_engine, String, Integer
from databricks.sqlalchemy import  DatabricksArray
from urllib.parse import quote_plus

df = pd.DataFrame([
    {
        "GROUP_ID": 1,
        "PARAM": ["abc@mail.com", "xyz@mail.com"],
    },
    {
        "GROUP_ID": 2,
        "PARAM": ["test@mail.com"],
    },
])

connection_string = (
    f"databricks://token:{quote_plus(TOKEN)}@{DATABRICKS_HOST}?"
    f"http_path={quote_plus(DATABRICKS_HTTP_PATH)}"
    f"&catalog={quote_plus(CATALOG)}"
    f"&schema={quote_plus(SCHEMA)}"
)

engine = create_engine(connection_string, echo=True)

df.to_sql(
    "mailing_group",
    con=engine,
    index=False,
    if_exists="replace",
    dtype={
        "GROUP_ID": Integer,
        "PARAM": DatabricksArray(String),
    },
)

Logs:
2026-08-11 08:15:07,657 INFO sqlalchemy.engine.Engine
CREATE TABLE mailing_group (
GROUP_ID INT,
PARAM ARRAY
) USING DELTA
TBLPROPERTIES('delta.feature.allowColumnDefaults' = 'enabled')
2026-08-11 08:15:07,657 INFO sqlalchemy.engine.Engine [no key 0.00074s] {}
2026-08-11 08:15:09,665 INFO sqlalchemy.engine.Engine INSERT INTO mailing_group (GROUP_ID, PARAM) VALUES (:GROUP_ID, :PARAM)
2026-08-11 08:15:09,666 INFO sqlalchemy.engine.Engine [generated in 0.00110s] [{'GROUP_ID': 1, 'PARAM': ['abc@mail.com', 'xyz@mail.com']}, {'GROUP_ID': 2, 'PARAM': ['test@mail.com']}]

Environment

1. Python : 3.12
2. Pandas : 2.2.3
3. databricks-sqlalchemy : 2.0.10
4. DBSQL Version: 2026.20
5. databricks-sql-connector:  4.0.5