Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Complete Slurm proxy health integration, draining, telemetry, and overload validation

オープン
  • 何か問題がありますか？
#1,103 コメント 0 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@nabinchha がすでに取り組んでいます。

2026年10月28日 から。

  • #1104 @nabinchha による — オープン

評価

この issue はまだ評価されていません。

説明

task

Task Summary

Complete the remaining lifecycle integration, observability, and live overload validation from #966. The core streaming, pooled inference proxy was delivered in #967; this issue carries forward the outstanding requirements so #966 can close without losing them.

Related: #1005 (disconnect cancellation and timeout fixes), #1044 (startup readiness and lifecycle diagnostics), #870 (sealed release acceptance).

Current behavior

  • Streaming, connection reuse, deterministic least-connections routing, bounded failover, aggregate backend health, and 429/Retry-After handling are implemented.
  • Startup readiness probes and required-process exit detection exist. During generation, HTTP backend health is not periodically connected to persisted managed readiness when a backend stays alive but becomes unhealthy.
  • The proxy configures a 10-second shutdown timeout. Runtime cleanup sends TERM to all managed steps, waits 0.2 seconds, then escalates to KILL; serving steps are not kept available until proxy draining completes.
  • Existing metrics include requests, attempts, per-backend active/request counts, connection opens/reuse/failures, retry reasons, and final status counts. Request logs include attempt count, upstream outcome sequence, final status, failover usage, and overall elapsed time.
  • Automated tests cover queue admission, 429 failover/header handling, streaming, connection reuse, and failure paths. Live vLLM overload and the resulting client AIMD behavior remain unverified; this is a validation gap, not a confirmed overload defect.

Remaining Work

Runtime health integration
  • Periodically check backend health after startup and propagate required-backend service loss to managed readiness and attempt state.
  • Cover a backend becoming unhealthy while its process remains alive, as well as process exit.
Coordinated shutdown
  • Stop admitting new proxy requests and allow active responses to drain for a bounded window while required backends remain available.
  • Coordinate runtime TERM/KILL escalation with that window, then close remaining sockets and stop serving steps.
  • Cover normal completion, cancellation, and failure.
Observability completion
  • Record the selected upstream for each attempt and separate connection and response timings.
  • Add connection eviction accounting and ensure terminal timeouts are observable, including when no further retry occurs.
  • Retain existing counters; final 429 and 5xx counts are already available through final-status counters.
  • Keep secrets, authorization headers, prompts, and generated content out of logs and metrics.
Live overload validation
  • Exercise actual queue saturation with partial and total replica overload.
  • Demonstrate final 429/Retry-After propagation after bounded failover and the client's admission reduction and cooldown.
  • Record sanitized code/artifact identities, expected and actual output counts, failure/retry counts, elapsed time, records per minute, and proxy CPU/memory telemetry. Coordinate release-artifact evidence with #870.

Acceptance Criteria

  • Required-backend health loss after startup is reflected in persisted managed readiness and the defined attempt-state behavior, even while its process remains alive.
  • Cleanup permits active proxy responses to complete within the configured drain budget before stopping their backends, then terminates remaining requests/sockets after the deadline.
  • Regression tests cover runtime health loss and coordinated draining on completion, cancellation, and failure.
  • Sanitized request telemetry identifies attempted upstreams and connection/response timings; eviction and terminal-timeout accounting is available.
  • Live overload evidence demonstrates bounded failover, end-to-end 429/Retry-After propagation, and the client's AIMD/cooldown response.
  • Evidence distinguishes automated/loopback coverage from live-cluster and sealed-artifact acceptance.

Source References

Assessment against feat/slurm-execution on 2026-10-28:

主要言語
Python
スター
2.4k
フォーク
241
平均マージ
1日 1時間
マージ済み PR（30日）
86

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA-NeMo/DataDesigner のほかの issue

NVIDIA-NeMo/DataDesigner の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。