Error Messages with Unity Catalog

Unity Catalog が有効なクラスターと有効でないクラスターに対して、SQLAlchemy create_engine と execute の 2 つのケースを再現し、レポートに示されているコネクターエラーを比較します。Databricks ServerOperationError がどこで表面化するかを追跡し、有効でないクラスターがカタログプラグインエラーではなく、より明確な UC_NOT_ENABLED メッセージを生成することを確認します。

On older versions of the connector (version 2.5.2) and connecting to a cluster where unity catalog is not enabled, it would give us the first failure message. However, after updating to the connector (version 2.9.3) and connecting to unity catalog enabled cluster, it's giving us a different error message, which doesn't help figure out what the issue is.

from sqlalchemy import create_engine

# Unity Catalog enabled
uc_eng = create_engine(f"databricks://{CONNECTION_STRING}?catalog=prod", connect_args={'http_path': f'sql/protocolv1/o/{WORKSPACE_ID}/{UC_CLUSTER_ID}'})
uc_eng.execute(f"select * from {TABLE_NAME} limit 1").fetchall()  # works

# NOT Unity Catalog ENABLED 
non_uc_eng = create_engine(f"databricks://{CONNECTION_STRING}?catalog=prod", connect_args={'http_path': f'sql/protocolv1/o/{WORKSPACE_ID}/{NON_UC_CLUSTER_ID}'})
non_uc_eng.execute("select * from {TABLE_NAME} limit 1").fetchall(). # fails 


Failure message:
DatabaseError: (databricks.sql.exc.ServerOperationError) Catalog 'prod' plugin class not found: spark.sql.catalog.prod is not defined
[SQL: select * from prod.{TABLE_NAME} limit 1]
(Background on this error at: http://sqlalche.me/e/13/4xp6)
instead of former:
sqlalchemy.exc.DatabaseError: (databricks.sql.exc.ServerOperationError) [UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster.