apache/airflow
Vedi su GitHubProvide configuration option to configure path of OTEL metrics exporter
Open
#42.492 aperta il 26 set 2024
good first issuekind:featuretelemetry
Metriche repository
- Star
- (44.809 star)
- Metriche merge PR
- (Merge medio 7g 18h) (834 PR mergiate in 30 g)
Descrizione
Description
Currently Airflow does not allow to configure path in OTEL metrics exporter
def get_otel_logger(cls) -> SafeOtelLogger:
host = conf.get("metrics", "otel_host") # ex: "breeze-otel-collector"
port = conf.getint("metrics", "otel_port") # ex: 4318
prefix = conf.get("metrics", "otel_prefix") # ex: "airflow"
ssl_active = conf.getboolean("metrics", "otel_ssl_active")
# PeriodicExportingMetricReader will default to an interval of 60000 millis.
interval = conf.getint("metrics", "otel_interval_milliseconds", fallback=None) # ex: 30000
debug = conf.getboolean("metrics", "otel_debugging_on")
resource = Resource(attributes={SERVICE_NAME: "Airflow"})
protocol = "https" if ssl_active else "http"
endpoint = f"{protocol}://{host}:{port}/v1/metrics"
And some of the metrics collectors, e.g. VictoriaMetrics expect the metrics to be sent to different path
/opentelemetry/api/v1/push/insert/0/opentelemetry/api/v1/push
It will be great to have the metrics path to be configurable, like the following
def get_otel_logger(cls) -> SafeOtelLogger:
host = conf.get("metrics", "otel_host") # ex: "breeze-otel-collector"
port = conf.getint("metrics", "otel_port") # ex: 4318
path = conf.get("metrics", "otel_path", fallback="/v1/metrics") # ex: "/opentelemetry/api/v1/push"
prefix = conf.get("metrics", "otel_prefix") # ex: "airflow"
ssl_active = conf.getboolean("metrics", "otel_ssl_active")
# PeriodicExportingMetricReader will default to an interval of 60000 millis.
interval = conf.getint("metrics", "otel_interval_milliseconds", fallback=None) # ex: 30000
debug = conf.getboolean("metrics", "otel_debugging_on")
resource = Resource(attributes={SERVICE_NAME: "Airflow"})
protocol = "https" if ssl_active else "http"
endpoint = f"{protocol}://{host}:{port}{path}"
Use case/motivation
Ability to send metrics directly into OTEL-compatible metrics collectors, e.g. VictoriaMetrics.
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct