Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Add a performance budget regression check to CI using the existing Prometheus metrics

Open
  • Is something wrong?
#247 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
github-actions, nodejs, prometheus, typescript
Domain
ci-cd, observability, performance

Research direction

Start with .github/workflows/build-check.yml to see the current steps, then read PROFILING.md (around lines 96-99) and performance-baseline.service.ts to find where baseline numbers would come from. The pure analytics functions in performance-analytics.service.ts are the suggested benchmark target. Done means a committed baseline, a deterministic benchmark, and a CI failure when a regression exceeds the stated tolerance. The scope is still open, so confirm the approach with the maintainers before starting.

Written by the indexing model from the issue text.

Description

advanced help wanted performance tooling

Problem

The repository has a capable performance instrumentation stack — @willsoto/nestjs-prometheus, prom-client, OpenTelemetry, src/observability/, src/profiling/, and a performance-baseline.service.ts that "establishes baseline performance on startup" and "continuously compares current metrics against baseline" (per PROFILING.md:96-99).

None of it is enforced. .github/workflows/build-check.yml runs codegen, tsc --noEmit, a narrow Jest subset, and npm run build — no performance step. PROFILING.md documents the tooling but no baseline numbers, targets, or budget exist, and there is no coverage threshold either (see the separate coverage issue).

ROADMAP.md sets "Reach 80% code coverage" and "Achieve 99.9% uptime in staging" as v0.2.0 goals with no mechanism to measure either.

Why it matters

Performance regressions are invisible without a gate. A change that adds an N+1 or drops a cache passes every existing check — type checking, lint, and the small test subset all succeed while latency quietly doubles. Since this is exactly the failure mode most of the performance issues in this batch describe, a budget is what stops them recurring.

The instrumentation is already paid for; nothing consumes it.

Constraints in the current setup

Two things to be careful about:

  • The CI test filter is narrow (src/graphql plus two spec files), so a performance test that needs broad coverage would run very little.
  • Running real load in CI is slow and flaky. Prefer a deterministic micro-benchmark over a throughput test.

Suggested Direction

  1. Record a baseline first. Use PROFILING.md's tooling to capture current p50/p95 for a small set of representative endpoints, and commit those numbers as the budget. Without a recorded baseline there is nothing to compare against.
  2. Add a deterministic benchmark step, not a load test. autocannon is already used by test/portfolio/portfolio-management.loadtest.sh, but CI-friendlier options exist. A small set of micro-benchmarks over pure functions — the analytics calculations in performance-analytics.service.ts are ideal, since they are deterministic and take no I/O.
  3. Assert the budget in CI. A regression beyond a stated tolerance (e.g. 20%) fails the build. Keep the tolerance generous initially; the goal is catching order-of-magnitude regressions, not 5% noise.
  4. Export the Prometheus metrics from the build-check job as an artifact, or run the service briefly and scrape /metrics, so PR-level latency is visible without deploying.
  5. Track the ROADMAP goal by reporting coverage and budget status on each run, so the 80% target becomes measurable.

Constraints

  • Benchmarks in CI are notoriously flaky on shared runners. Pin to a fixed workload size, discard warmup iterations, and take the median of several runs rather than a single measurement.
  • Do not make CI fail on absolute latency numbers alone; relative regression against a committed baseline is more stable across runner types.
  • Keep the benchmark suite fast — it runs on every PR.
  • Do not let the benchmark depend on network or a live database.

Acceptance Criteria

  • A baseline of representative metrics is recorded and committed.
  • At least one deterministic benchmark covers a hot code path.
  • CI fails on a regression beyond the stated tolerance.
  • The benchmark is stable across repeated runs on the CI runner.
  • Prometheus metrics are exported as a CI artifact.
  • The benchmark completes within a reasonable PR budget (document the actual time).
Dominant language
TypeScript
Stars
7
Forks
55
Avg merge
1d 17h
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from SourceXXL/alian_structure-api

All issues in SourceXXL/alian_structure-api

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.