apache/airflow

Add SRV DNS record support to http connections

オープン

#9,186 opened on 2020/06/08

 (9 件のコメント) (0 件のリアクション) (0 人の担当者)Python (16,781 件のフォーク)batch import
good first issuekind:featureprovider:http

Repository metrics

Stars
 (44,809 個のスター)
PR merge metrics
 (PR metrics pending)

説明

Description

SRV records are DNS records that resolve to a hostname and a port. Being able to specify http connections with SRV resolution support is useful in contexts where external services are defined as SRV records.

Use case / motivation

One particular example applies to service discovery with dynamic port mapping. In this context, service discovery maps one or more hosts to a SRV record. When resolving that SRV record a client gets one or more hosts and ports to reach the desired service. In my particular use case I am using AWS Service Discovery for ECS with dynamic port mapping. I would like to reach my ECS service transparently from Airflow without going thru a load balancer.

I would like to be able to specify an http connection with a SRV record and let Airflow resolve the actual hostname and port using SRV lookup.

Related Issues

https://github.com/apache/airflow/pull/4481

I'll be happy to provide the implementation for this

コントリビューターガイド