apache/airflow

Add SRV DNS record support to http connections

Open

#9.186 aberto em 8 de jun. de 2020

Ver no GitHub
 (9 comments) (0 reactions) (0 assignees)Python (16.781 forks)batch import
good first issuekind:featureprovider:http

Métricas do repositório

Stars
 (44.809 stars)
Métricas de merge de PR
 (Mesclagem média 7d 18h) (834 fundiu PRs em 30d)

Description

Description

SRV records are DNS records that resolve to a hostname and a port. Being able to specify http connections with SRV resolution support is useful in contexts where external services are defined as SRV records.

Use case / motivation

One particular example applies to service discovery with dynamic port mapping. In this context, service discovery maps one or more hosts to a SRV record. When resolving that SRV record a client gets one or more hosts and ports to reach the desired service. In my particular use case I am using AWS Service Discovery for ECS with dynamic port mapping. I would like to reach my ECS service transparently from Airflow without going thru a load balancer.

I would like to be able to specify an http connection with a SRV record and let Airflow resolve the actual hostname and port using SRV lookup.

Related Issues

https://github.com/apache/airflow/pull/4481

I'll be happy to provide the implementation for this

Guia do colaborador