Record on the actor when a resume moved it to another node, and prefer a node with the image when detaching

オープン
#3,482 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
gcp, go, kubernetes

調査の方向性

The issue does not name files or tests; start by locating the resume/detach scheduling path and the actor status and atelet restore-record definitions. Trace how a resumed actor is assigned to a worker and how its source node is available. Done means detaches prefer a node that already has the image when it has room, and the actor status and restore record capture the move and source node.

索引モデルが issue の本文から書いたものです。

説明

area/api area/node area/scheduling kind/feature
What happens today

A suspended actor is pinned to its node. Since the detach change, if that node is full at wake time the resume moves the actor to any worker with room. That is the right behavior. Two things around it could be better.

Today Suggestion
Visibility the actor's status keeps the old node; the only sign of a move is a snapshot download in atelet's restore record record the move and the source node on the actor status
Destination chosen without looking at images, so the new node usually unpacks the image (~28 s here) prefer a node that already holds the image, the same preference #276 proposes for placement

In a density run, 36,631 of 105,400 wakes moved, and I could only count them by joining logs afterwards.

Proposal

Detach prefers a worker on a node that holds the image when one has room (see the placement proposal). The actor status records that the resume moved and from which node. atelet's restore record carries the node name.

How to see it
  1. A fleet of suspended actors that fits the slots, plus a second workload that keeps creating fresh actors so the slots on the fleet's nodes are taken between its steps.
  2. Resume the fleet. In atelet's restore records, the resumes with download above zero and seconds of oci_unpack are the ones that moved; nothing on the actor says so.
Setup

etup: two GKE worker pools. Pool A: 40 x n2-standard-8 (8 vCPU, 32 GiB, Intel Cascade Lake), 192 GB pd-balanced boot disk, 320 worker pods. Pool B: 783 x e2-standard-8 (8 vCPU, 32 GiB, AMD Rome), 500 GB pd-balanced boot disk, 2,000 worker pods. ateapi at 2 to 12 replicas; Postgres on one n2-standard-16 with pd-ssd. Images: SWE-bench Verified (445, about 1.1 GB compressed median, 13 layers), R2E-Gym-V1 (8,101, 0.57 GB compressed median, 1.9 GB max, 14 layers, almost no layer sharing), a 36,871-image SWE-RL set (0.5 to 1.4 GB compressed median per subset, up to 13 GB squashed single-layer images); unpacked size 2 to 2.5 times compressed; plain containerd pulls from a same-region Artifact Registry. gVisor (runsc), GKE 1.36.4-gke.2046002, Container-Optimized OS, Linux x86_64. Observed on Substrate 0fdb67b5; the code cited is unchanged on main at a508f8fe.

主要言語
Go
スター
5.9k
フォーク
637
平均マージ
2日 1時間
マージ済み PR(30日)
523

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

agent-substrate/substrate のほかの issue

agent-substrate/substrate の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。