apache/airflow

JdbcHook doesn't support OpenLineage

Geschlossen

#41.878 geöffnet am 30.08.2024

 (13 Kommentare) (0 Reaktionen) (0 zugewiesene Personen)Python (16.781 Forks)batch import
area:providersgood first issuekind:featureprovider:openlineage

Repository-Metriken

Stars
 (44.809 Sterne)
PR-Merge-Metriken
 (Durchschn. Merge 7T 18h) (834 gemergte PRs in 30 T)

Beschreibung

Apache Airflow Provider(s)

common-sql

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.24.0 apache-airflow-providers-celery==3.7.2 apache-airflow-providers-cncf-kubernetes==8.3.1 apache-airflow-providers-common-compat==1.2.0 apache-airflow-providers-common-io==1.3.2 apache-airflow-providers-common-sql==1.14.0 apache-airflow-providers-docker==3.12.0 apache-airflow-providers-elasticsearch==5.4.1 apache-airflow-providers-fab==1.1.1 apache-airflow-providers-ftp==3.9.1 apache-airflow-providers-google==10.19.0 apache-airflow-providers-grpc==3.5.1 apache-airflow-providers-hashicorp==3.7.1 apache-airflow-providers-http==4.11.1 apache-airflow-providers-imap==3.6.1 apache-airflow-providers-jdbc==4.3.1 apache-airflow-providers-microsoft-azure==10.1.1 apache-airflow-providers-microsoft-mssql==3.7.1 apache-airflow-providers-microsoft-winrm==3.5.1 apache-airflow-providers-mongo==4.1.1 apache-airflow-providers-mysql==5.6.1 apache-airflow-providers-odbc==4.6.1 apache-airflow-providers-openlineage==1.11.0 apache-airflow-providers-postgres==5.11.1 apache-airflow-providers-redis==3.7.1 apache-airflow-providers-samba==4.7.1 apache-airflow-providers-sendgrid==3.5.1 apache-airflow-providers-sftp==4.10.1 apache-airflow-providers-slack==8.7.1 apache-airflow-providers-smtp==1.7.1 apache-airflow-providers-snowflake==5.5.1 apache-airflow-providers-sqlite==3.8.1 apache-airflow-providers-ssh==3.11.1

Apache Airflow version

2.9.2

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Docker-Compose

Deployment details

No response

What happened

There is no support of OpenLineage in JdbcHook. For example to extract metadata and create table and columns in openlineage. We find this https://github.com/OpenLineage/OpenLineage/blob/main/integration/airflow/openlineage/airflow/extractors/sql_extractor.py but unfortunatly we can't used it

What you think should happen instead

No response

How to reproduce

Always the case

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor Guide