expfmt: TextParser cost grows quadratically with the number of labels on a sample
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- go, yaml
- Domain
- data-engineering
Research direction
The quadratic duplicate check is in startLabelName in the expfmt text parser (expfmt/text_parse.go in prometheus/common). Read that function first, then the label-reading loop that calls it for each sample. Done means one set is kept per sample with one insert per label name, and the existing text parser tests still pass, plus a benchmark showing the 40-label gauge gets cheaper.
Written by the indexing model from the issue text.
Description
The text parser checks each label name of a sample for duplicates in startLabelName. At each label it builds a new map of all the labels read so far, so a sample with n labels costs n maps and n(n+1)/2 inserts: 136 for 16 labels, 820 for 40.
Info-style metrics often carry many labels. vLLM exposes vllm:cache_config_info as one sample with 40 labels, and a scraper built on this parser parses it on every scrape. In llm-d's endpoint picker, which parses a few families from each vLLM endpoint every 50 ms, this line is half of the scrape CPU.
TextParser.TextToMetricFamilies on v0.74.0 with model.UTF8Validation, a new parser per iteration, Go 1.27.0 on darwin/arm64 (Apple M5), medians of six runs:
| input | time | allocated | allocations |
|---|---|---|---|
| gauge, 1 label | 1.0 µs | 5.4 KB | 36 |
| gauge, 16 labels | 6.9 µs | 13.5 KB | 173 |
| gauge, 40 labels | 35.5 µs | 73.0 KB | 487 |
vllm:cache_config_info from vLLM 0.34 (HELP, TYPE and the 40-label sample) |
41 µs | 73.6 KB | 492 |
vLLM 0.34 /metrics page, 643 lines |
614 µs | 589 KB | 16,301 |
the same page without vllm:cache_config_info |
553 µs | 520 KB | 15,818 |
Above the 1-label line, going from 16 to 40 labels multiplies the time by 5.9, close to the 6.0 ratio of the insert counts. In a CPU profile of the 40-label gauge, startLabelName takes 86% of the time spent in TextToMetricFamilies and 90% of the allocated bytes. On the full vLLM page, the one 40-label line accounts for 12% of the bytes the parser allocates.
Keeping one set per sample and adding each label name as it is read would make the check one insert per label.
- Dominant language
- Go
- Stars
- 297
- Forks
- 380
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 12
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from prometheus/common
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
prometheus/common#1023 ·
Maintainers usually reply within 2 days
-
expfmt: OpenMetrics 1.0 output with a unit that is not a name suffix fails the whole scrape in PrometheusPossibly taken @itsmeasadali claimed this 42 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 58/100
prometheus/common#1005 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 52/100
prometheus/common#876 · 1 comment ·
Maintainers usually reply within 2 days
-
Feature request: Add client_assertion field for oauth2 jwt-bearer grant typePossibly taken @alliasgher claimed this 220 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 68/100
prometheus/common#869 ·
Maintainers usually reply within 2 days
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
prometheus/common#858 ·
Maintainers usually reply within 2 days
All issues in prometheus/common
Similar issues
-
Difficulty 2/5 Half a day Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
keyxmakerx/Chronicle#2616 ·
Maintainers usually reply within 1 day
-
tech-debt
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
internal/mcp: an exported-but-empty env var is reported as configured but never injectedPossibly taken A pull request linked to this issue is open or already merged. Opengo good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day