Error Messages with Unity Catalog
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
Research direction
Reproduce the two SQLAlchemy create_engine and execute cases against Unity Catalog-enabled and non-enabled clusters, comparing the connector errors shown in the report. Trace where the Databricks ServerOperationError is surfaced and verify that a non-enabled cluster produces the clearer UC_NOT_ENABLED message instead of the catalog plugin error.
Written by the indexing model from the issue text.
Description
On older versions of the connector (version 2.5.2) and connecting to a cluster where unity catalog is not enabled, it would give us the first failure message. However, after updating to the connector (version 2.9.3) and connecting to unity catalog enabled cluster, it's giving us a different error message, which doesn't help figure out what the issue is.
from sqlalchemy import create_engine
# Unity Catalog enabled
uc_eng = create_engine(f"databricks://{CONNECTION_STRING}?catalog=prod", connect_args={'http_path': f'sql/protocolv1/o/{WORKSPACE_ID}/{UC_CLUSTER_ID}'})
uc_eng.execute(f"select * from {TABLE_NAME} limit 1").fetchall() # works
# NOT Unity Catalog ENABLED
non_uc_eng = create_engine(f"databricks://{CONNECTION_STRING}?catalog=prod", connect_args={'http_path': f'sql/protocolv1/o/{WORKSPACE_ID}/{NON_UC_CLUSTER_ID}'})
non_uc_eng.execute("select * from {TABLE_NAME} limit 1").fetchall(). # fails
Failure message:
DatabaseError: (databricks.sql.exc.ServerOperationError) Catalog 'prod' plugin class not found: spark.sql.catalog.prod is not defined
[SQL: select * from prod.{TABLE_NAME} limit 1]
(Background on this error at: http://sqlalche.me/e/13/4xp6)
instead of former:
sqlalchemy.exc.DatabaseError: (databricks.sql.exc.ServerOperationError) [UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster.
- Dominant language
- Python
- Stars
- 233
- Forks
- 152
- Avg merge
- 21h 5m
- Merged PRs (30d)
- 10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/databricks-sql-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
All issues in databricks/databricks-sql-python
Similar issues
-
essnmx good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
-
[Feature] 奇物选择添加优先级 Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
syfoud/Simulated_Scepter#174 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Giskard-AI/giskard-oss#2840 · 1 comment ·
-
A claim comment carrying the issue number is silently declined while the workflow reports success Openarea: repo bug perceived difficulty: 2
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
yeti-platform/yeti#1380 ·