apache/airflow

Add functionality to DatabricksHook to create notebooks

Chiusa

#22.330 aperta il 17 mar 2022

Vedi su GitHubGuida contributor
 (7 commenti) (0 reazioni) (0 assegnatari)Python (16.781 fork)batch import
area:providersgood first issuekind:featureprovider:databricks

Metriche repository

Star
 (44.809 stelle)
Metriche merge PR
 (Merge medio 7g 18h) (834 PR mergiate in 30 g)

Descrizione

Description

Currently, there is no way to programmatically create a notebook in Databricks using the core provider components (DatabricksHook, DatabricksRunNowOperator, or DatabricksSubmitRunOperator). If implemented, this change would allow Airflow users to create a Scala, R, Python, or SQL notebook in DBFS programmatically from Airflow.

Use case/motivation

This would be useful for our users who are hosting their notebooks in an Airflow repository and would like to utilize advantages of tools like jinja templating, orchestration, git version control, etc.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guida contributor

Direzione di ricerca
Examine the existing DatabricksHook in airflow/providers/databricks/hooks/databricks.py to understand the current API and how to add a method for creating notebooks. Look at the Databricks REST API documentation for notebook creation endpoints. Review any linked PRs or comments for additional context. The implementation should create a new method in DatabricksHook that calls the Databricks API to create a notebook in DBFS.
Tech stack
python
Dominio
backend
Tipo issue
Funzionalità
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
3
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
1-3 ore
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
Nuova
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
Chiara
Prerequisiti
Basic understanding of DatabricksFamiliarity with Airflow hooks and operators
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
55