apache/airflow

JdbcHook doesn't support OpenLineage

Chiusa

#41.878 aperta il 30 ago 2024

 (13 commenti) (0 reazioni) (0 assegnatari)Python (16.781 fork)batch import
area:providersgood first issuekind:featureprovider:openlineage

Metriche repository

Star
 (44.809 stelle)
Metriche merge PR
 (Merge medio 7g 18h) (834 PR mergiate in 30 g)

Descrizione

Apache Airflow Provider(s)

common-sql

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.24.0 apache-airflow-providers-celery==3.7.2 apache-airflow-providers-cncf-kubernetes==8.3.1 apache-airflow-providers-common-compat==1.2.0 apache-airflow-providers-common-io==1.3.2 apache-airflow-providers-common-sql==1.14.0 apache-airflow-providers-docker==3.12.0 apache-airflow-providers-elasticsearch==5.4.1 apache-airflow-providers-fab==1.1.1 apache-airflow-providers-ftp==3.9.1 apache-airflow-providers-google==10.19.0 apache-airflow-providers-grpc==3.5.1 apache-airflow-providers-hashicorp==3.7.1 apache-airflow-providers-http==4.11.1 apache-airflow-providers-imap==3.6.1 apache-airflow-providers-jdbc==4.3.1 apache-airflow-providers-microsoft-azure==10.1.1 apache-airflow-providers-microsoft-mssql==3.7.1 apache-airflow-providers-microsoft-winrm==3.5.1 apache-airflow-providers-mongo==4.1.1 apache-airflow-providers-mysql==5.6.1 apache-airflow-providers-odbc==4.6.1 apache-airflow-providers-openlineage==1.11.0 apache-airflow-providers-postgres==5.11.1 apache-airflow-providers-redis==3.7.1 apache-airflow-providers-samba==4.7.1 apache-airflow-providers-sendgrid==3.5.1 apache-airflow-providers-sftp==4.10.1 apache-airflow-providers-slack==8.7.1 apache-airflow-providers-smtp==1.7.1 apache-airflow-providers-snowflake==5.5.1 apache-airflow-providers-sqlite==3.8.1 apache-airflow-providers-ssh==3.11.1

Apache Airflow version

2.9.2

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Docker-Compose

Deployment details

No response

What happened

There is no support of OpenLineage in JdbcHook. For example to extract metadata and create table and columns in openlineage. We find this https://github.com/OpenLineage/OpenLineage/blob/main/integration/airflow/openlineage/airflow/extractors/sql_extractor.py but unfortunatly we can't used it

What you think should happen instead

No response

How to reproduce

Always the case

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Guida contributor