apache/airflow

Integrate Snowflake Notebook Scheduling with Apache Airflow

Open

#40.167 geöffnet am 10. Juni 2024

Auf GitHub ansehen
 (14 Kommentare) (9 Reaktionen) (0 zugewiesene Personen)Python (16.781 Forks)batch import
area:providersgood first issuekind:featureprovider:snowflake

Repository-Metriken

Stars
 (44.809 Stars)
PR-Merge-Metriken
 (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)

Beschreibung

Description

Enable scheduling and execution of Snowflake Notebooks from Apache Airflow. This feature would allow users to orchestrate and manage Snowflake Notebooks directly within Airflow, leveraging Airflow's scheduling, dependency management, and monitoring capabilities.

Use case/motivation

The recent launch of Snowflake Notebooks offers a powerful environment for data engineering, analytics, and machine learning workflows. By integrating Snowflake Notebooks with Airflow, users can automate their data workflows more efficiently. This integration will enable users to:

  • Schedule regular execution of Snowflake Notebooks.
  • Incorporate Snowflake Notebooks into larger, multi-step data pipelines.
  • Use Airflow's monitoring and alerting features to manage the execution of Snowflake Notebooks.
  • Reduce the need for manual intervention, thereby increasing productivity and ensuring consistency in data processes.

This feature will greatly benefit data teams who already use Snowflake and Airflow, providing a seamless and automated way to manage their end-to-end data workflows.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor Guide