Plan preview request routing scans all active pipeds

Aperta
  • Qualcosa non va?
#7,087 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

  • #7088 di @rootp1 — chiusa senza merge

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
go
Ambito
api, performance

Direzione di ricerca

Inizia in pkg/app/server/grpcapi/api.go, in API.RequestPlanPreview, e analizza la chiamata a pipedStore.List e il ciclo di corrispondenza dei repository. Aggiungi un benchmark che copra quantità piccole e grandi di piped, quindi valuta una ricerca mirata, un indice o una cache limitata per le richieste ripetute. Il lavoro è completo quando il routing ripetuto evita le scansioni complete dei piped attivi, preservando al contempo risultati corretti quando corrispondono più piped.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happened:

RequestPlanPreview currently lists all enabled pipeds in the project and scans every configured repository entry in memory to find pipeds that match the requested repository remote and base branch. This makes each interactive plan preview request scale with total active piped count and repository mappings instead of the number of matching pipeds.

Summary

Plan preview request routing performs an O(number of pipeds x repositories per piped) in-memory scan for every request.

Affected Area

pkg/app/server/grpcapi/api.go, specifically API.RequestPlanPreview.

Steps to Reproduce

  1. Create a test or benchmark environment with many active pipeds in one project and repository mappings on those pipeds.
  2. Call API.RequestPlanPreview repeatedly for one repository remote and base branch.
  3. Capture allocations and latency with go test -bench or pprof.
  4. Observe that request cost grows with total piped count rather than with the matching pipeds.

What you expected to happen:

Plan preview routing should avoid a full active-piped scan on every request. Repeated routing for a repository/base-branch pair should be close to O(matches), for example via an index, bounded cache, or targeted lookup.

Expected Behaviour

Routing a plan preview request should avoid repeated full scans and should keep behavior correct when multiple pipeds match the same repository and branch.

How to reproduce it:

  1. Seed or mock thousands of enabled pipeds for one project.
  2. Give only a subset a repository entry matching the plan preview request remote and base branch.
  3. Repeatedly call RequestPlanPreview for that remote and branch.
  4. Compare latency/allocations as piped count increases.

Actual Behaviour

Each request calls pipedStore.List with only ProjectId and Disabled=false filters, then iterates through every returned piped and every repository entry until it finds matching mappings.

Technical Evidence

In pkg/app/server/grpcapi/api.go, RequestPlanPreview lists enabled pipeds for the project and then builds a repositories map by checking each p.Repositories entry against req.RepoRemoteUrl and req.BaseBranch. The code also has a TODO noting that caching may be needed to reduce database load.

Suspected Root Cause

The API layer has no repository-to-piped routing index or cache, so it must perform repository matching after fetching all enabled pipeds.

Proposed Direction

Add a focused benchmark for plan preview routing at small and large piped cardinalities, then avoid repeated full scans by adding a bounded cache or another targeted lookup for repository-to-piped matches.

Acceptance Criteria

  • A benchmark covering small and large piped counts is added.
  • Repeated plan preview routing for the same project/repository/base-branch avoids repeated full active-piped scans.
  • Correct behavior is preserved when multiple pipeds match the same repository and branch.

Environment:

  • piped version: Not applicable
  • control-plane version: Current master branch
  • Others: Not applicable
Lingua principale
Go
Stelle
1.4k
Fork
391
Merge medio
4g 1h
PR unite (30g)
58

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pipe-cd/pipecd

Tutte le issue di pipe-cd/pipecd

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.