Liquid clustering support for Databricks tables
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start by reviewing the Alembic and SQLAlchemy migration flow described in the issue, including generated migration files and the use of op.execute for Databricks DDL. Determine how native CLUSTER BY and liquid clustering should be represented and how autogenerate would emit them; done means the recurring manual ALTER TABLE step is covered by an agreed implementation and tests.
Written by the indexing model from the issue text.
Description
We recently implemented Alembic for schema migration management on Databricks and came across a major concern-
There is no native CLUSTER BY support
Alembic and SQLAlchemy have no concept of Databricks-specific DDL like CLUSTER BY
Every migration file generated by autogenerate needs to be manually edited to add op.execute("ALTER TABLE ... CLUSTER BY (...)")
This is a recurring manual step that is easy to forget and can't be automated
Is there a roadmap for native CLUSTER BY and liquid clustering support and if not, can this be considered to be implemented?
- Dominant language
- Python
- Stars
- 24
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/databricks-sqlalchemy
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
databricks/databricks-sqlalchemy#73 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
All issues in databricks/databricks-sqlalchemy
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
xinnan-tech/xiaozhi-fde-talk#263 ·
-
rules
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
huggingface/Repo2RLEnv#163 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 95/100
huggingface/sentence-transformers#4074 ·
-
comp/dashboard invalid P3
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
NousResearch/hermes-agent#121143 ·