connection refused for kafka transport
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
Research direction
Start with the Kafka transport configuration and broker connection handling in the repository, then check how the provided Docker Compose setup configures Kafka listeners and startup readiness. Reproduce the connection refused error using the reported compose configuration and identify whether the broker address or startup sequence causes it. Done means the transport connects successfully or the issue includes a confirmed cause and a clear configuration fix.
Written by the indexing model from the issue text.
Description
Hi,
I am having this issue with the code while trying to use : confluentinc/cp-kafka
Error :
time=2025-10-01T05:35:12.167Z level=ERROR msg="error transporter" error="kafka: client has run out of available brokers to talk to: dial tcp 172.20.0.3:9092: connect: connection refused for kafka transport"
docker-compose. File:
`services:
kafka:
image: confluentinc/cp-kafka:latest
ports:
- 9092:9092
environment:
KAFKA_KRAFT_MODE: "true" # Enables KRaft mode.
KAFKA_PROCESS_ROLES: controller,broker # Kafka acts as both controller and broker.
KAFKA_NODE_ID: 1 # Unique ID for the Kafka instance.
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
KAFKA_ADVERTISED_LISTENERS: "BROKER://kafka:9092"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,BROKER:PLAINTEXT,IB:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: BROKER
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_LOG_DIRS: /var/lib/kafka/data # Log storage location.
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" # Enables automatic topic creation.
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 # Single replica for simplicity.
KAFKA_LOG_RETENTION_HOURS: 168 # Log retention period (7 days).
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 # No rebalance delay.
CLUSTER_ID: AAAAAAAAAAAAAAAAAAAAAA
KAFKA_ENABLE_KRAFT: yes
# dd if=/dev/zero of=/dev/stdout count=16 bs=1 | openssl enc -A -a
# head -c 20 /dev/zero | base64
restart: always
grafana:
image: grafana/grafana:12.1
environment:
- GF_INSTALL_PLUGINS=vertamedia-clickhouse-datasource
# - GF_INSTALL_PLUGINS=grafana-clickhouse-datasource
# - GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=vertamedia-clickhouse-datasource
ports:
- 3000:3000
restart: always
volumes:
- ./grafana/datasources-ch.yml:/etc/grafana/provisioning/datasources/datasources-ch.yml
- ./grafana/dashboards.yml:/etc/grafana/provisioning/dashboards/dashboards.yml
- ./grafana/dashboards:/var/lib/grafana/dashboards
prometheus:
image: prom/prometheus:v3.6.0
ports:
- 9090:9090
restart: always
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
goflow2:
build:
context: ../../
dockerfile: Dockerfile
args:
VERSION: compose
LDFLAGS: -X main.version=compose
image: netsampler/goflow2
depends_on:
- kafka
ports:
- 8080:8080
- 6343:6343/udp
- 2055:2055/udp
restart: always
command:
- -transport.kafka.brokers=kafka:9092
- -transport=kafka
- -transport.kafka.topic=flows
- -format=bin
db:
image: clickhouse/clickhouse-server:24.11.1.2557-alpine
ports:
- 8123:8123
volumes:
- ./clickhouse:/docker-entrypoint-initdb.d/
#- ../../pb/flow.proto:/var/lib/clickhouse/format_schemas/flow.proto
# the following files have 101:101 as owner
- ./clickhouse/flow.proto:/var/lib/clickhouse/format_schemas/flow.proto
- ./clickhouse/protocols.csv:/var/lib/clickhouse/user_files/protocols.csv
depends_on:
- kafka`
- Dominant language
- Go
- Stars
- 817
- Forks
- 182
- Avg merge
- 4h 18m
- Merged PRs (30d)
- 6
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from netsampler/goflow2
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
netsampler/goflow2#601 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
netsampler/goflow2#539 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
netsampler/goflow2#528 ·
Maintainers usually reply within 1 day
-
NetFlow v9: zero OUT_BYTES/OUT_PKTS (FreeBSD ng_netflow/OPNsense) overwrite real IN_BYTES/IN_PKTS — bytes always 0Possibly taken @devMaxMcCann claimed this 8 days ago. Open
Difficulty 3/5 Half a day Newbie friendliness 52/100
netsampler/goflow2#524 · 2 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 35/100
netsampler/goflow2#523 · 2 comments ·
Maintainers usually reply within 1 day
All issues in netsampler/goflow2
Similar issues
-
enhancement exporter/awskinesis needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
open-telemetry/opentelemetry-collector-contrib#53334 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
erigontech/erigon#26523 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 83/100
SocialGouv/iterion#3376 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
seaweedfs/seaweedfs#12740 · 2 comments ·
Maintainers usually reply within 1 day
-
area:indexer bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
seanb4t/codegraph-go#385 ·
Maintainers usually reply within 1 day