NetFlow v9 IN_BYTES/OUT_BYTES and IN_PKTS/OUT_PKTS overwrite each other

Open Beginner friendly
#601 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
networking

Research direction

Read producer/proto/producer_nf.go, especially ConvertNetFlowDataSet and the Statistics switch block named in the issue. Build a test input from the listed NetFlow v9 field order and check how IN and OUT counters are mapped, taking care to distinguish v9 OUT fields from IPFIX types 23 and 24. Done when a regression test covers the reported zero OUT counters and the resulting flow reports the expected bytes and packets.

Written by the indexing model from the issue text.

Description

Repository: netsampler/goflow2. Version: v2.2.6. File: producer/proto/producer_nf.go, function
ConvertNetFlowDataSet (the switch df.Type block under "Statistics", lines 324-339).

Summary

ConvertNetFlowDataSet maps the v9 counters by assignment:

case netflow.NFV9_FIELD_IN_BYTES:  DecodeUNumber(v, &(flowMessage.Bytes))
case netflow.NFV9_FIELD_IN_PKTS:   DecodeUNumber(v, &(flowMessage.Packets))
case netflow.NFV9_FIELD_OUT_BYTES: DecodeUNumber(v, &(flowMessage.Bytes))
case netflow.NFV9_FIELD_OUT_PKTS:  DecodeUNumber(v, &(flowMessage.Packets))

Both the IN and the OUT counter write the same field, so the field that comes last in the template wins.
An exporter that lists the OUT counters after the IN counters and sends 0 in them makes every flow report
0 bytes and 0 packets.

Reproduction

Exporter: OPNsense (FreeBSD ng_netflow), NetFlow v9, template 256 (IPv4) and 259 (IPv6). The template
field order is:

IPV4_SRC_ADDR, IPV4_DST_ADDR, IPV4_NEXT_HOP, INPUT_SNMP, OUTPUT_SNMP, IN_PKTS, IN_BYTES, OUT_PKTS, OUT_BYTES, FIRST_SWITCHED, LAST_SWITCHED, L4_SRC_PORT, L4_DST_PORT, TCP_FLAGS, PROTOCOL, SRC_TOS, SRC_AS, DST_AS, SRC_MASK, DST_MASK

Records on the wire (decoded by hand): IN_BYTES=256 IN_PKTS=1 OUT_BYTES=0 OUT_PKTS=0,
IN_BYTES=52 IN_PKTS=1 OUT_BYTES=0 OUT_PKTS=0. goflow2 reports Bytes=0 and Packets=0 for both.

Expected

The counters of a flow are the traffic of the flow. Either add the IN and OUT counters of one record, or
ignore a 0 value in the later field. Adding is what a bidirectional record means; ignoring zero is the
smaller change.

Notes

  • IPFIX types 23 (postOctetDeltaCount) and 24 (postPacketDeltaCount) share the numbers 23 and 24 with
    the v9 OUT counters. They count the same traffic after a middlebox, so a fix that adds them for IPFIX
    would count the traffic twice. A fix needs to separate v9 and IPFIX there.
  • We carry a local patch for the v9 case (adds the OUT counter to the IN counter of one record before the
    producer runs) and would drop it once this is fixed upstream.
  • A test input is easy to build from the field order above; no capture is needed.
Dominant language
Go
Stars
817
Forks
182
Avg merge
4h 18m
Merged PRs (30d)
6

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from netsampler/goflow2

All issues in netsampler/goflow2

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.