Plan preview request routing scans all active pipeds

Abierto
  • ¿Algo no está bien?
#7,087 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

  • #7088 de @rootp1 — cerrado sin fusionar

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
go
Área
api, performance

Línea de trabajo

Comienza en pkg/app/server/grpcapi/api.go, en API.RequestPlanPreview, e inspecciona la llamada a pipedStore.List y el bucle de coincidencia de repositorios. Añade un benchmark que cubra cantidades pequeñas y grandes de pipeds; después, evalúa una búsqueda específica, un índice o una caché acotada para solicitudes repetidas. Se considera terminado cuando el enrutamiento repetido evita los escaneos completos de los pipeds activos y conserva los resultados correctos cuando coinciden varios pipeds.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

What happened:

RequestPlanPreview currently lists all enabled pipeds in the project and scans every configured repository entry in memory to find pipeds that match the requested repository remote and base branch. This makes each interactive plan preview request scale with total active piped count and repository mappings instead of the number of matching pipeds.

Summary

Plan preview request routing performs an O(number of pipeds x repositories per piped) in-memory scan for every request.

Affected Area

pkg/app/server/grpcapi/api.go, specifically API.RequestPlanPreview.

Steps to Reproduce

  1. Create a test or benchmark environment with many active pipeds in one project and repository mappings on those pipeds.
  2. Call API.RequestPlanPreview repeatedly for one repository remote and base branch.
  3. Capture allocations and latency with go test -bench or pprof.
  4. Observe that request cost grows with total piped count rather than with the matching pipeds.

What you expected to happen:

Plan preview routing should avoid a full active-piped scan on every request. Repeated routing for a repository/base-branch pair should be close to O(matches), for example via an index, bounded cache, or targeted lookup.

Expected Behaviour

Routing a plan preview request should avoid repeated full scans and should keep behavior correct when multiple pipeds match the same repository and branch.

How to reproduce it:

  1. Seed or mock thousands of enabled pipeds for one project.
  2. Give only a subset a repository entry matching the plan preview request remote and base branch.
  3. Repeatedly call RequestPlanPreview for that remote and branch.
  4. Compare latency/allocations as piped count increases.

Actual Behaviour

Each request calls pipedStore.List with only ProjectId and Disabled=false filters, then iterates through every returned piped and every repository entry until it finds matching mappings.

Technical Evidence

In pkg/app/server/grpcapi/api.go, RequestPlanPreview lists enabled pipeds for the project and then builds a repositories map by checking each p.Repositories entry against req.RepoRemoteUrl and req.BaseBranch. The code also has a TODO noting that caching may be needed to reduce database load.

Suspected Root Cause

The API layer has no repository-to-piped routing index or cache, so it must perform repository matching after fetching all enabled pipeds.

Proposed Direction

Add a focused benchmark for plan preview routing at small and large piped cardinalities, then avoid repeated full scans by adding a bounded cache or another targeted lookup for repository-to-piped matches.

Acceptance Criteria

  • A benchmark covering small and large piped counts is added.
  • Repeated plan preview routing for the same project/repository/base-branch avoids repeated full active-piped scans.
  • Correct behavior is preserved when multiple pipeds match the same repository and branch.

Environment:

  • piped version: Not applicable
  • control-plane version: Current master branch
  • Others: Not applicable
Lenguaje dominante
Go
Estrellas
1.4k
Forks
391
Merge medio
4 d 1 h
PR fusionados (30 d)
58

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de pipe-cd/pipecd

Todos los issues de pipe-cd/pipecd

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.