Support connection with Proxy in Databricks provider
#32 576 ouverte le 13 juil. 2023
Métriques du dépôt
- Stars
- (44 809 étoiles)
- Métriques de merge PR
- (Merge moyen 7j 18h) (834 PRs mergées en 30 j)
Description
Apache Airflow version
2.6.3
What happened
Databricks connection is unable to query login.microsoftonline.com if using a proxy as there is no provision in databricks.py file to include proxy as part of requests.post command to include proxy in it.
What you think should happen instead
there is no provision in databricks.py file to include proxy as part of requests.post command to include proxy in it.
How to reproduce
Try reaching login.microsoftonline.com endpoint via proxy. Tried providing proxy as part of extra in databricks connection { "azure_tenant_id": "xxxx", "proxies": { "http": "http://xxxx:443", "https": "http://xxxx:443" } } Error: {databricks_base.py:434} INFO - Using AAD Token for SPN. {databricks_base.py:223} INFO - Existing AAD token is expired, or going to expire soon. Refreshing... {connectionpool.py:1003} DEBUG - Starting new HTTPS connection (1): login.microsoftonline.com:443 {databricks_base.py:469} ERROR - Attempt 1 API Request to Databricks failed with reason: <Future at 0x7f1954c4e280 state=finished raised ConnectTimeout>
Noticed that line 251 where the requests.post happens doesn't have this enabled.
Operating System
Linux
Versions of Apache Airflow Providers
No response
Deployment
Other
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct