influxdata/telegraf

CloudWatch Metrics Input Plugin to Support Percentile Based Metrics

Open

#7,951 opened on Aug 6, 2020

View on GitHub
 (6 comments) (1 reaction) (0 assignees)Go (9,892 stars) (4,161 forks)batch import
area/awscloudfeature requesthelp wantedsize/m

Description

Feature Request

Opening a feature request kicks off a discussion.

Proposal:

The CloudWatch GetMetricData API call has the capability to use an Expression to allow for gathering pXX.XX (IE p95) statistics metrics.

Current behavior:

Based on this function, only 'minimum, maximum, sum, samplecount, average' are supported metric values. There is no extendedmetrics available which would be used by extended metrics.

Desired behavior:

Allow for pXX.XX to be valid metrics to be gathered by the CloudWatch input plugin.

Use case:

For statistical analysis, it is often better to throw out outliers of a dataset to keep from distracting from underlying trends. Using p95 allows for a higher confidence interval in what real "max" usage is, p5 similarly for "minimum" usage, and p50 for "average" usage. This allows for a statistics_input object to contain arbitrary pXX.XX values and return them from the GetMetricData API. This is suggested as an additional option, rather than complete replacement, to allow for users of this plugin to choose which statistic set they prefer.

Contributor guide

Tech stack
goaws
Domain
cloudbackend
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
GoAWS CloudWatch APIPlugin architecture
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
The issue requests extending the CloudWatch input plugin to support percentile based statistics (e.g., p95). The current implementation in plugins/inputs/cloudwatch/cloudwatch.go only supports minimum, maximum, sum, sample count, and average. To implement, one must modify the `getStatistics` or similar function to accept extended statistics like pXX.XX. The AWS SDK's `GetMetricData` API supports metric expressions, so the plugin needs to allow configuration of percentile values. The discussion in the issue and the linked code provide the starting point. No existing pull requests address this.