Catalog creation on Databricks Free

Open
  • Is something wrong?
#5,695 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
go
Domain
cli, databases

Research direction

Start with the catalog creation request at POST /api/2.1/unity-catalog/catalogs and the YAML configuration shown in the issue. Compare the behavior when storage_root is omitted with Databricks Free's Default Storage requirement, then verify that catalog creation succeeds without an explicit storage location.

Written by the indexing model from the issue text.

Description

DABs
Describe the issue

DABs should support catalogs creation. One of the required parameters is storage_root. However on Databricks Free, I can't set this as I don't have access to underlying storage.

Configuration
bundle:
  name: poc
  engine: direct

workspace:
  host: 
  root_path: /Workspace/Users/[email protected]/.bundle/${bundle.target}/catalogs/${bundle.name}

targets:
  prod:
    mode: production
    default: true

resources:
  catalogs:
    poc:
      name: poc
      comment: "Proof of concept catalog for development purposes"
      lifecycle:
        prevent_destroy: true
Steps to reproduce the behavior
  1. Run databricks bundle deploy ...
  2. See error
Expected Behavior

There should be an option to create catalog without specifying storage_root parameter.

Actual Behavior
Uploading bundle files to /Workspace/Users/[email protected]/.bundle/prod/catalogs/poc/files...
Deploying resources...
Error: cannot create resources.catalogs.poc: Metastore storage root URL does not exist. Default Storage is enabled in your account. You can use the UI to create a new catalog using Default Storage, or please provide a storage location for the catalog (for example 'CREATE CATALOG myCatalog MANAGED LOCATION '<location-path>'). (400 INVALID_STATE)

Endpoint: POST https://....cloud.databricks.com/api/2.1/unity-catalog/catalogs
HTTP Status: 400 Bad Request
API error_code: INVALID_STATE
API message: Metastore storage root URL does not exist. Default Storage is enabled in your account. You can use the UI to create a new catalog using Default Storage, or please provide a storage location for the catalog (for example 'CREATE CATALOG myCatalog MANAGED LOCATION '<location-path>').
OS and CLI version

OS: Fedora
CLI: v1.4.0

Is this a regression?

n/a - trying for the first time.

Dominant language
Go
Stars
427
Forks
267
Avg merge
2d 1h
Merged PRs (30d)
233

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/cli

All issues in databricks/cli

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.