apache/airflow

Add SRV DNS record support to http connections

Open

#9 186 ouverte le 8 juin 2020

Voir sur GitHub
 (9 commentaires) (0 réactions) (0 assignés)Python (16 781 forks)batch import
good first issuekind:featureprovider:http

Métriques du dépôt

Stars
 (44 809 stars)
Métriques de merge PR
 (Merge moyen 7j 18h) (834 PRs mergées en 30 j)

Description

Description

SRV records are DNS records that resolve to a hostname and a port. Being able to specify http connections with SRV resolution support is useful in contexts where external services are defined as SRV records.

Use case / motivation

One particular example applies to service discovery with dynamic port mapping. In this context, service discovery maps one or more hosts to a SRV record. When resolving that SRV record a client gets one or more hosts and ports to reach the desired service. In my particular use case I am using AWS Service Discovery for ECS with dynamic port mapping. I would like to reach my ECS service transparently from Airflow without going thru a load balancer.

I would like to be able to specify an http connection with a SRV record and let Airflow resolve the actual hostname and port using SRV lookup.

Related Issues

https://github.com/apache/airflow/pull/4481

I'll be happy to provide the implementation for this

Guide contributeur