out_datadog: support Datadog Workload Identity Federation (AWS) instead of a static apikey

Aperta
  • Qualcosa non va?
#12,715 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
aws, c
Ambito
backend, cloud

Direzione di ricerca

Start by reading the out_datadog plugin and its AWS credential resolution through flb_aws_credentials, then compare the delegated authentication flow with the linked Datadog Agent implementation. The issue describes the required STS signing, intake-key request, and refresh behavior, but does not name Fluent Bit files or tests. Done means an opt-in AWS mode obtains and refreshes a managed key for log intake.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Is your feature request related to a problem? Please describe.

The Datadog output plugin (out_datadog) only authenticates with a static apikey. Datadog now supports Workload Identity Federation for AWS, so the Datadog Agent can authenticate with the AWS credentials of its environment instead of a long-lived API key. Fluent Bit doesn't support this.

This matters most on ECS Fargate. Datadog's recommended log path there is FireLens with out_datadog, and the Agent can't collect container logs on Fargate. Teams that move their Agents to WIF still have to keep a static API key in every task definition just for the log router.

Describe the solution you'd like

An opt-in mode in out_datadog that gets its API key from Datadog using the AWS credentials Fluent Bit already resolves (flb_aws_credentials: env, ECS task role, EKS IRSA/Pod Identity, IMDS). It would mirror the Agent's delegated_auth flow (datadog-agent#46272, datadog-agent#46291):

  1. Build a SigV4-signed STS GetCallerIdentity request with the x-ddog-org-id: <org UUID> header signed in.
  2. POST https://api.<site>/api/v2/intake-key with Authorization: Delegated <base64 body>|<base64 headers>|POST|<base64 STS URL>.
  3. Use the returned managed API key for log intake, and refresh it before it expires or after a 403.

Example config:

[OUTPUT]
    Name               datadog
    Match              *
    Host               http-intake.logs.datadoghq.eu
    TLS                on
    delegated_auth     aws
    dd_org_uuid        <org-uuid>
    aws_region         eu-west-1

Describe alternatives you've considered

  • Keep a static apikey in the FireLens secretOptions. This works, but it's the long-lived credential WIF is meant to remove.
  • Send logs to the Datadog Agent sidecar over TCP so only the Agent talks to Datadog. Datadog doesn't document this on Fargate, it turns the Agent into a single point of failure for logs, and per-container service/source tagging has to move into Agent config.
  • Use CloudWatch Logs plus the Datadog Forwarder Lambda. The forwarder also needs a static API key.

Additional context

We run dozens of ECS Fargate services that use FireLens with out_datadog, and we're moving all Datadog authentication to WIF. With out_datadog support we could remove the last static Datadog API key from our task definitions. The Agent's AWS implementation is in comp/core/delegatedauth.

Lingua principale
C
Stelle
8.2k
Fork
2k
Merge medio
3g 20h
PR unite (30g)
99

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di fluent/fluent-bit

Tutte le issue di fluent/fluent-bit

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.