elastic/kibana

[Infrastructure UI] Anomaly Detection job for network doesn't consider `system.network.name`

Open

#221,133 opened on May 21, 2025

View on GitHub
 (3 comments) (0 reactions) (0 assignees)TypeScript (19,065 stars) (8,021 forks)batch import
Feature:ObsHostsTeam:obs-presentationbuggood first issue

Description

Describe the bug:

The anomaly detection job for Hosts includes 2 jobs for network traffic (RX & TX), the aggregation is doing a derivative on system.network.in.bytes and system.network.out.bytes but it's not considering that those counters are unique per system.network.name. The data for this is inaccurate, if you have 2+ network interfaces, it will use the highest counter instead of combining the counters.

Potential Fix:

Under the data_histogram, add a terms aggregation for system.network.name with a max sub aggregation on system.network.in.bytes. Use a sum_bucket pipeline aggregation to add the max values together then take the derivative.

Working Example:

Here is a working example that you can use in the Dev Console with Metricbeat data.

POST metrics-*/_search
{
  "size": "0",
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-15m",
              "lte": "now"
            }
          }
        }
      ]
    }
  },
  "aggregations": {
    "host.name": {
      "terms": {
        "field": "host.name"
      },
      "aggregations": {
        "host.name": {
          "terms": {
            "field": "host.name",
            "size": 100
          },
          "aggregations": {
            "buckets": {
              "date_histogram": {
                "field": "@timestamp",
                "fixed_interval": "5m"
              },
              "aggregations": {
                "@timestamp": {
                  "max": {
                    "field": "@timestamp"
                  }
                },
                "interfaces": {
                  "terms": {
                    "field": "system.network.name",
                    "size": 20
                  },
                  "aggregations": {
                    "max_bytes": {
                      "max": {
                        "field": "system.network.in.bytes"
                      }
                    }
                  }
                },
                "total_max_bytes": {
                  "sum_bucket": {
                    "buckets_path": "interfaces>max_bytes"
                  }
                },
                "bytes_in_derivative": {
                  "derivative": {
                    "buckets_path": "total_max_bytes"
                  }
                },
                "positive_only": {
                  "bucket_script": {
                    "buckets_path": {
                      "in_derivative": "bytes_in_derivative.value"
                    },
                    "script": "params.in_derivative > 0.0 ? params.in_derivative : 0.0"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Contributor guide