prometheus/prometheus

Turn one source label into multiple target labels

Open

#12,896 opened on Sep 27, 2023

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Go (64,042 stars) (10,408 forks)batch import
component/service discoveryhelp wantedkind/enhancementpriority/P3

Description

Proposal

Hey

So, we have the usecase that we have one central prometheus instance and developers do not have access to the prometheus configuration. We are scraping consul for our metrics targets. While applications can provide their own labels, there is also certain data that the application doesn't know of, for example its (dynamic) id in our deployment system. What we'd like to be able to do, is to have some generic consul tag where can extract promtheus labels from.

For example with this config:

      - source_labels: [__meta_consul_tags]
        regex: "(?:^|.*?,)prom_label__(.+?)=(.+?)(?:$|,.*?)"
        action: replace
        target_label: $1
        replacement: $2

Given these consul tags:

a=b,b=c,prom_label__f=g,t=e,prom_label__z=l,l=o

I'd expect these labels in prometheus for the given target:

f=g,z=l

The workaround would be to manually list the labels for all services, which could be quite a lot.

I've already looked at the code and guess that we can easily change the regex code to use multiple matches instead of just one set of submatches. Adjusting the tests is an easy feat as well. This should imo not be a breaking change.

Contributor guide

Tech stack
go
Domain
observability
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go programming basicsUnderstanding of Prometheus relabelingRegex knowledge
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75
Research direction
Examine the relabeling code in Prometheus, likely in the 'relabel' package, specifically the 'Process' function that handles 'replace' actions with source labels and regex. The current implementation only uses the first set of regex submatches; the goal is to iterate over all matches to generate multiple target labels. The tests in the same package (e.g., 'relabel test.go') need to be updated to cover the new behavior. Consider performance implications if many labels are generated. The issue provides a clear example of the desired behavior.
Turn one source label into multiple target labels · prometheus/prometheus#12896 | Good First Issue