Measured-only placement planner for worker pools (sie_placement)

Abierto
#903 4 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
helm, kubernetes, python

Línea de trabajo

Start by reading the workspace package layout and the existing worker pool, Helm chart, and KEDA trigger definitions; the issue names no specific files or tests. The proposed work spans versioned measurement schemas, a placement planner, a tuner, GPU availability, Helm output, an API applier, and chart integration. Done means delivering these in the eight staged pull requests while keeping the gateway pool specification and chart KEDA triggers unchanged.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

Operators size worker pools by hand today. A Helm worker pool per machine profile, minReplicas, the pool warm floor (minimum_worker_count) and pinned_models decide which models run on which GPUs and which stay warm. Each choice trades three things: how long a request waits after a model has been idle, what the warm capacity costs, and how fast the model answers once warm. Nothing in the repository states the target for those three or derives a pool layout from measurements.

Proposal

Add a measured-only placement planner as a new workspace package, sie_placement, standard library only, not published to PyPI at first.

  • Input documents, versioned JSON. Per served identity (a model id or model:profile) they hold a come-alive p95, a warm p99 (time to first token and inter-token latency for generation) and a queue-wait p99, with optional named tiers. Receipts are throughput and latency measurements per identity on a machine profile and the GPU product the measurement actually ran on. The other documents are boot evidence, a price list per machine profile, and hourly demand.
  • Planner core, a pure function. It keeps only measured cells that meet the warm bound, picks the cheapest warmth that meets the come-alive bound, and emits a plan with a per-lane envelope (floor, buffer and scale-down window ranges). Its report gives the cost of each service level, the next cheaper option and the bound that blocks it, and ranked measurements worth taking. Cells without measurements are never chosen.
  • Tuner step, a pure function that keeps a lane's floor inside its envelope from live queue signals.
  • Optional pieces: a plan-to-Helm-values emitter, a sie-placement tune applier that moves a lane's warm floor inside its envelope through an API logical pool, and an opt-in chart workload for it.

The gateway's pool specification and the chart's KEDA triggers do not change.

Delivery

Eight stacked pull requests: schemas, planner core, tuner step, GPU availability, several models on one GPU, Helm values, the applier, and chart integration.

Lenguaje dominante
Python
Estrellas
3.5k
Forks
328
Merge medio
16 h 25 min
PR fusionados (30 d)
121

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de superlinked/sie

Todos los issues de superlinked/sie

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.