Record on the actor when a resume moved it to another node, and prefer a node with the image when detaching

Abierto
  • ¿Algo no está bien?
#3,482 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
gcp, go, kubernetes
Área
backend, cloud, distributed-systems

Línea de trabajo

The issue does not name files or tests; start by locating the resume/detach scheduling path and the actor status and atelet restore-record definitions. Trace how a resumed actor is assigned to a worker and how its source node is available. Done means detaches prefer a node that already has the image when it has room, and the actor status and restore record capture the move and source node.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area/api area/node area/scheduling kind/feature
What happens today

A suspended actor is pinned to its node. Since the detach change, if that node is full at wake time the resume moves the actor to any worker with room. That is the right behavior. Two things around it could be better.

Today Suggestion
Visibility the actor's status keeps the old node; the only sign of a move is a snapshot download in atelet's restore record record the move and the source node on the actor status
Destination chosen without looking at images, so the new node usually unpacks the image (~28 s here) prefer a node that already holds the image, the same preference #276 proposes for placement

In a density run, 36,631 of 105,400 wakes moved, and I could only count them by joining logs afterwards.

Proposal

Detach prefers a worker on a node that holds the image when one has room (see the placement proposal). The actor status records that the resume moved and from which node. atelet's restore record carries the node name.

How to see it
  1. A fleet of suspended actors that fits the slots, plus a second workload that keeps creating fresh actors so the slots on the fleet's nodes are taken between its steps.
  2. Resume the fleet. In atelet's restore records, the resumes with download above zero and seconds of oci_unpack are the ones that moved; nothing on the actor says so.
Setup

etup: two GKE worker pools. Pool A: 40 x n2-standard-8 (8 vCPU, 32 GiB, Intel Cascade Lake), 192 GB pd-balanced boot disk, 320 worker pods. Pool B: 783 x e2-standard-8 (8 vCPU, 32 GiB, AMD Rome), 500 GB pd-balanced boot disk, 2,000 worker pods. ateapi at 2 to 12 replicas; Postgres on one n2-standard-16 with pd-ssd. Images: SWE-bench Verified (445, about 1.1 GB compressed median, 13 layers), R2E-Gym-V1 (8,101, 0.57 GB compressed median, 1.9 GB max, 14 layers, almost no layer sharing), a 36,871-image SWE-RL set (0.5 to 1.4 GB compressed median per subset, up to 13 GB squashed single-layer images); unpacked size 2 to 2.5 times compressed; plain containerd pulls from a same-region Artifact Registry. gVisor (runsc), GKE 1.36.4-gke.2046002, Container-Optimized OS, Linux x86_64. Observed on Substrate 0fdb67b5; the code cited is unchanged on main at a508f8fe.

Lenguaje dominante
Go
Estrellas
5.9k
Forks
637
Merge medio
2 d 1 h
PR fusionados (30 d)
523

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de agent-substrate/substrate

Todos los issues de agent-substrate/substrate

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.