Memory leak issue when running with systemd n send to kafka partition with key
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- go, kafka, ubuntu
- Domain
- backend, data-engineering, networking
Research direction
Start by examining the Kafka transport’s partition key handling and the NetFlow template processing path; the report gives no source files or tests to locate them. Reproduce with the supplied systemd and mapping configuration while replaying traffic, then check whether memory keeps growing, the Kafka partition key is correct, and template warnings stop.
Written by the indexing model from the issue text.
Description
Describe the bug
- Memory Leak
- kafka partition key coming as 8�(� in kafka
- few warning coming => Sep 06 13:36:01 ccs-122 goflow2[3961852]: time=2025-09-06T13:36:01.765Z level=WARN msg="template error" scheme=netflow hostname="" port=8057 count=1 workers=2 blocking=false queue_size=1000000
** Configs **
Systemd config
[Unit]
Description=GoFlow2 NetFlow Collector on port 8057
After=network.target
[Service]
ExecStart=/opt/netflow/bin/goflow2
-mapping=/opt/netflow/config/mapping_8057.yaml
-transport=kafka
-transport.kafka.brokers=10.10.10.122:9093
-transport.kafka.topic=netflow
-transport.kafka.hashing
-listen=netflow://:8057
-addr=:18057
WorkingDirectory=/opt/netflow
Restart=always
RestartSec=5
User=root
Group=root
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
cat /opt/netflow/config/mapping_8057.yaml
formatter:
type: json
key:
- sampler_address
mapping:
- output: sampler_address
value: "$sampler_address"
To Reproduce
Steps to reproduce the behavior:
- Run systemd service with systemctl commands
- run traffic using tcp replay or actual router via some other machine
- check : systemd-cgtop => memory keep growing
Expected behavior
A clear and concise description of what you expected to happen.
- kafka patition key should be correct
- no memory leak
GoFlow2:
- Version: latest: v2.2.3
- Environment: systemd
- OS: Ubuntu 22.04 jammy
- Dominant language
- Go
- Stars
- 817
- Forks
- 182
- Avg merge
- 4h 18m
- Merged PRs (30d)
- 6
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from netsampler/goflow2
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
netsampler/goflow2#601 ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
netsampler/goflow2#539 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
netsampler/goflow2#528 ·
Maintainers usually reply within 1 day
-
NetFlow v9: zero OUT_BYTES/OUT_PKTS (FreeBSD ng_netflow/OPNsense) overwrite real IN_BYTES/IN_PKTS — bytes always 0Possibly taken @devMaxMcCann claimed this 8 days ago. Open
Difficulty 3/5 Half a day Newbie friendliness 52/100
netsampler/goflow2#524 · 2 comments · 1 reaction ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 35/100
netsampler/goflow2#523 · 2 comments ·
Maintainers usually reply within 1 day
All issues in netsampler/goflow2
Similar issues
-
enhancement exporter/awskinesis needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
open-telemetry/opentelemetry-collector-contrib#53334 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
erigontech/erigon#26523 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 83/100
SocialGouv/iterion#3376 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
seaweedfs/seaweedfs#12740 · 2 comments ·
Maintainers usually reply within 1 day
-
area:indexer bug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
seanb4t/codegraph-go#385 ·
Maintainers usually reply within 1 day