SPLUNK_HOSTNAME variable doesn't set the hostname

Reproduce the issue with the supplied compose YAML and .env file using the Splunk Universal Forwarder 9.4.12 image. Start by tracing how SPLUNK_HOSTNAME is handled during container startup and compare it with SPLUNK_BEFORE_START_CMD=set default-hostname REDACTED. Done means the environment variable sets the hostname, or the documented workaround is made explicit.

I tried looking in /etc/hostname, a btool of server and inputs configs, and I can't find the value I set for SPLUNK_HOSTNAME. This is using the version 9.4.12 of the UF image on dockerhub, here's my redacted config file:

splunk:
  role: splunk_universal_forwarder
  service_name: SplunkForwarder
  conf:
    - key: inputs
      value:
        directory: /opt/splunkforwarder/etc/system/local
        content:
          default:
            _meta: "org_id::CHANGEME"
    - key: server
      value:
        directory: /opt/splunkforwarder/etc/system/local
        content:
          deployment:
            pass4SymmKey: REDACTED

And my ENV file:

DEPLOYMENT_SYMM_REPLACE=REDACTED
SPLUNK_BEFORE_START_CMD=add forward-server REDACTED:9997 -client-cert /mnt/certs/cert.pem -ssl-root-ca-path /mnt/certs/cacert.pem -ssl-verify-server-cert true
SPLUNK_PASSWORD=REDACTED
SPLUNK_HOSTNAME=REDACTED
DEBUG=true

And my compose yaml:

version: '3.8'

services:
  busybox-app:
    image: busybox:latest
    container_name: busybox-app
    command: >
      sh -c '
        mkdir -p /var/log/busybox &&
        counter=1 &&
        while true; do
          echo "$$(date +"%Y-%m-%d %H:%M:%S") [ID:$$counter] Test message unique string buffalo" >> /var/log/busybox/busy.log;
          counter=$$((counter + 1));
          sleep 5;
        done
      '
    volumes:
      - shared-logs:/var/log/busybox
    restart: unless-stopped

  splunk-forwarder:
    image: splunk/universalforwarder:9.4.12
    container_name: splunk-forwarder
    env_file: ./.env
    environment:
      SPLUNK_ADD: "monitor /var/log/busybox/busy.log -sourcetype generic_single_line -index main"
      SPLUNK_START_ARGS: --accept-license
    volumes:
      - shared-logs:/var/log/busybox
      # Only needed if overwriting config
      - ./configs.yml:/tmp/defaults/default.yml
      - ./certs/:/mnt/certs/
    depends_on:
      - busybox-app
    restart: unless-stopped

volumes:
  shared-logs:

For people looking at this in the future, the workaround is to use the splunk set default-hostname command in your env file:

SPLUNK_BEFORE_START_CMD=set default-hostname REDACTED