[Infrastructure UI] Anomaly Detection job for network doesn't consider `system.network.name`
#221,133 opened on May 21, 2025
Description
Describe the bug:
The anomaly detection job for Hosts includes 2 jobs for network traffic (RX & TX), the aggregation is doing a derivative on system.network.in.bytes and system.network.out.bytes but it's not considering that those counters are unique per system.network.name. The data for this is inaccurate, if you have 2+ network interfaces, it will use the highest counter instead of combining the counters.
Potential Fix:
Under the data_histogram, add a terms aggregation for system.network.name with a max sub aggregation on system.network.in.bytes. Use a sum_bucket pipeline aggregation to add the max values together then take the derivative.
Working Example:
Here is a working example that you can use in the Dev Console with Metricbeat data.
POST metrics-*/_search
{
"size": "0",
"query": {
"bool": {
"filter": [
{
"range": {
"@timestamp": {
"gte": "now-15m",
"lte": "now"
}
}
}
]
}
},
"aggregations": {
"host.name": {
"terms": {
"field": "host.name"
},
"aggregations": {
"host.name": {
"terms": {
"field": "host.name",
"size": 100
},
"aggregations": {
"buckets": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "5m"
},
"aggregations": {
"@timestamp": {
"max": {
"field": "@timestamp"
}
},
"interfaces": {
"terms": {
"field": "system.network.name",
"size": 20
},
"aggregations": {
"max_bytes": {
"max": {
"field": "system.network.in.bytes"
}
}
}
},
"total_max_bytes": {
"sum_bucket": {
"buckets_path": "interfaces>max_bytes"
}
},
"bytes_in_derivative": {
"derivative": {
"buckets_path": "total_max_bytes"
}
},
"positive_only": {
"bucket_script": {
"buckets_path": {
"in_derivative": "bytes_in_derivative.value"
},
"script": "params.in_derivative > 0.0 ? params.in_derivative : 0.0"
}
}
}
}
}
}
}
}
}
}