[Bug]: A template without resource limits is admitted with no resource check, and a worker without limits registers the node's allocatable as its capacity

オープン
  • 何か問題がありますか？
#3,479 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
go, kubernetes
領域
backend-api-design, infrastructure

調査の方向性

Start by tracing template create/derive validation, WorkerPool worker-limit handling, scheduler admission checks, and worker capacity registration; the issue does not name specific files or tests. Review the API guide alongside those entry points to understand the documented per-worker budget. Done means missing limits are handled consistently, registered capacity is logged, and the API guide explains the behavior.

索引モデルが issue の本文から書いたものです。

説明

area/api-machinery area/scheduling kind/bug
What happened?

Two defaults combine so that no admission control runs, with nothing reporting it.

Default Effect
A template with empty resources.limits is accepted (older templates have it; derived templates inherit it) the scheduler skips the resource check for actors that declare no size; the only bound left is the worker's --max-actors, default 1000
A WorkerPool without limits on the worker container the worker reads its capacity through the downward API, which returns the node's allocatable; every worker on the node registers the whole node
Expected Behavior

A template without limits is refused at create and derive time, or receives a documented default. A WorkerPool without limits is refused, or the controller sets worker limits from the pool spec. The registered capacity is logged at worker registration. The API guide says what happens when either is missing.

Steps to Reproduce

My cluster ran 890 and then 512 concurrent actors on 320 workers for a week with admission never engaging. Each of the eight workers on an n2-standard-8 had registered 7.5 CPU, 28.9 GiB and 1,000 actors as its own capacity. The API guide describes a per-worker budget that declared sizes are booked against, and that is what I assumed was happening.

Suggestion
  • Refuse a template without limits, or apply a documented default.
  • Refuse a WorkerPool without worker limits, or derive them.
  • Log the capacity a worker registers.
Sandbox Runtime

gVisor (runsc)

Agent Substrate Version / Commit SHA

0fdb67b5

Kubernetes Version & Environment

GKE 1.36.4-gke.2046002, Container-Optimized OS, us-central1

Host OS & Architecture

Linux x86_64

Relevant Logs and Diagnostic Output
# a derived template
resources:
  limits: {}

# a worker on an n2-standard-8 (8 workers on the node, each reports this)
status:
  capacity:
    actors: 1000
    resources:
      limits:
      - name: cpu
        quantity: 7510m
      - name: memory
        quantity: 28927020Ki
  state: WORKER_STATE_ACTIVE
Additional Context

Setup: two GKE worker pools. Pool A: 40 x n2-standard-8 (8 vCPU, 32 GiB, Intel Cascade Lake), 192 GB pd-balanced boot disk, 320 worker pods (8 per node). Pool B: 783 x e2-standard-8 (8 vCPU, 32 GiB, AMD Rome), 500 GB pd-balanced boot disk, 2,000 worker pods. ateapi at 2 to 12 replicas; Postgres on one n2-standard-16 with pd-ssd. Images: SWE-bench Verified (445 images, about 1.1 GB compressed median, 13 layers), R2E-Gym-V1 (8,101 images, 0.57 GB compressed median, 1.9 GB max, 14 layers, almost no layer sharing between images), and a 36,871-image SWE-RL set (0.5 to 1.4 GB compressed median per subset, up to 13 GB squashed single-layer images); unpacked size is 2 to 2.5 times compressed. All images pulled from an Artifact Registry repository in the same region with plain containerd pulls. Load: create N actors from per-image ActorTemplates with golden snapshots enabled, send one command each, delete; or keep a fleet and suspend/resume it per step. Observed on 0fdb67b5; the code involved is unchanged on main at a508f8fe.

Confirmation
  • I have searched existing issues and verified that this is not a duplicate.
  • I have verified that this issue occurs on the latest commit on main.
主要言語
Go
スター
5.9k
フォーク
637
平均マージ
2日 1時間
マージ済み PR（30日）
523

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

agent-substrate/substrate のほかの issue

agent-substrate/substrate の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。