out_datadog: support Datadog Workload Identity Federation (AWS) instead of a static apikey

Abierto
  • ¿Algo no está bien?
#12,715 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
aws, c
Área
backend, cloud

Línea de trabajo

Start by reading the out_datadog plugin and its AWS credential resolution through flb_aws_credentials, then compare the delegated authentication flow with the linked Datadog Agent implementation. The issue describes the required STS signing, intake-key request, and refresh behavior, but does not name Fluent Bit files or tests. Done means an opt-in AWS mode obtains and refreshes a managed key for log intake.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Is your feature request related to a problem? Please describe.

The Datadog output plugin (out_datadog) only authenticates with a static apikey. Datadog now supports Workload Identity Federation for AWS, so the Datadog Agent can authenticate with the AWS credentials of its environment instead of a long-lived API key. Fluent Bit doesn't support this.

This matters most on ECS Fargate. Datadog's recommended log path there is FireLens with out_datadog, and the Agent can't collect container logs on Fargate. Teams that move their Agents to WIF still have to keep a static API key in every task definition just for the log router.

Describe the solution you'd like

An opt-in mode in out_datadog that gets its API key from Datadog using the AWS credentials Fluent Bit already resolves (flb_aws_credentials: env, ECS task role, EKS IRSA/Pod Identity, IMDS). It would mirror the Agent's delegated_auth flow (datadog-agent#46272, datadog-agent#46291):

  1. Build a SigV4-signed STS GetCallerIdentity request with the x-ddog-org-id: <org UUID> header signed in.
  2. POST https://api.<site>/api/v2/intake-key with Authorization: Delegated <base64 body>|<base64 headers>|POST|<base64 STS URL>.
  3. Use the returned managed API key for log intake, and refresh it before it expires or after a 403.

Example config:

[OUTPUT]
    Name               datadog
    Match              *
    Host               http-intake.logs.datadoghq.eu
    TLS                on
    delegated_auth     aws
    dd_org_uuid        <org-uuid>
    aws_region         eu-west-1

Describe alternatives you've considered

  • Keep a static apikey in the FireLens secretOptions. This works, but it's the long-lived credential WIF is meant to remove.
  • Send logs to the Datadog Agent sidecar over TCP so only the Agent talks to Datadog. Datadog doesn't document this on Fargate, it turns the Agent into a single point of failure for logs, and per-container service/source tagging has to move into Agent config.
  • Use CloudWatch Logs plus the Datadog Forwarder Lambda. The forwarder also needs a static API key.

Additional context

We run dozens of ECS Fargate services that use FireLens with out_datadog, and we're moving all Datadog authentication to WIF. With out_datadog support we could remove the last static Datadog API key from our task definitions. The Agent's AWS implementation is in comp/core/delegatedauth.

Lenguaje dominante
C
Estrellas
8.2k
Forks
2k
Merge medio
3 d 20 h
PR fusionados (30 d)
99

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de fluent/fluent-bit

Todos los issues de fluent/fluent-bit

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.