`host` argument value in docker compose setting

Start with the reduced Docker Compose YAML, the fluentd/fluentd.conf configuration, and test_server/async_fluentd.py. Review the linked Fluent Logger Python issues and container networking guidance to determine what the host argument should represent. Done means documenting a verified Compose setup, including the relevant service-to-service connection details and any Kubernetes limitations.

hey there,

Problem

i'm trying to test this out in a docker compose setting, but struggling to get the fluentd service to receive the logs being sent by the application using the fluentd logger in another service. reduced docker compose yaml:

Approach

services:

  test-server:
    profiles: [fluentd]
    build:
      context: ./test_server
      dockerfile: Dockerfile
    image: fluentd-logging-server
    command: [python, /home/fluent-logging/test_server/test_server async_fluentd.py]
    environment:
      LOG_FILE: /home/fluent-logging/logs/dummy-input-server.log
      TAG: log.server # populates `tag` positional argument of the fluentd async handler
      HOST: localhost # populates `host` keyword argument of the fluentd async handler
      PORT: 24224 # populates `port` keyword argument of the fluentd async handler
    volumes:
      - type: bind
        source: ./
        target: /home/fluent-logging

  fluentd:
    profiles: [fluentd]
    build:
      context: ./fluentd
      dockerfile: Dockerfile
    image: fluentd-logging-daemon
    command: fluentd -c /etc/fluent/fluent.conf -v
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    volumes:
      - type: bind
        source: ./fluentd/fluentd.conf
        target: /etc/fluent/fluent.conf

reduced fluentd config:

<source>
  @type forward
  port          24224
  <parse>
    @type json
  </parse>
</source>

<filter log.server>
  @type record_transformer
  <record>
    log_framework fluentd
    service_name  fluentd-test
    test          true
  </record>
</filter>

<match log.server>
  @type stdout
</match>

Context

The aim is to eventually deploy this on K8s, but that seems questionable at this point: see here and here. Fluentd's official doc also only mention this library in a non-containerized localhost scenario.