Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

A frozen node is twice as expensive as an absent one, and marks its peer's completed work as failed

Đang mở
  • Có gì không ổn?
#6 7 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 5 ngày

@Noah-Tervalon-Nvidia đang làm issue này rồi.

Từ ngày 11/9/2026.

Đánh giá

Issue này chưa được đánh giá.

Mô tả

Two observations from the same failure mode, measured on a two-node cluster (Ubuntu + RTX 3060, macOS, v0.1.1-463). Both matter for a cluster that includes a laptop, since a laptop that sleeps mid-request is exactly this shape: the socket is accepted, nothing answers.

Setup

24 concurrent chat/completions, max_tokens: 200, google/gemma-4-12b resident on both nodes.

Condition, same load Wall clock
Both nodes healthy 37.8 s
Single node, bypassing the router entirely 72.8 s
Peer frozen (SIGSTOP on its nvpair processes) 155.6 s

Every request returned 200 in all three. Reliability is not the problem — the cost is.

A frozen peer is 2.1× worse than not having it at all, and 4.1× worse than a healthy cluster. When I instead killed the peer's processes outright, the same load finished in 28.3 s with no failures: a closed socket gives an immediate RST and the proxy moves on in milliseconds. It is specifically the accepted-but-silent socket that hurts.

For context on the eviction path: after the peer was killed, discovery had dropped it from the node list within 75 s — I sampled too coarsely to pin it more precisely. During that window each request pays its own connection timeout.

The second half: false failures

When the frozen peer resumed, its job history marked several of the other node's workloads as

Failed — node serveri stopped reporting this workload

Those workloads had all completed successfully; the requesting node returned 200 for all 24. The peer's view had simply gone stale while it was stopped, and on resume it reaped them as failed. The timestamps match the resume to the second.

That leaves the job history — the only place a user sees what the cluster did — showing failures that did not happen, precisely in the situation where something unusual was going on and they would be looking at it.

Not filed as a defect

Neither is a crash and both may be deliberate. But if a laptop is a first-class node, asleep with the socket accepted and nothing answering is its most likely state, not an edge case. Two directions that would help, if either is welcome:

  • a connection-level timeout distinctly shorter than the request timeout, so an unresponsive candidate is abandoned quickly while a slow-but-live generation is not
  • reaping a peer's workloads as unknown rather than failed when the gap was on our side

Happy to open a PR for either if you have a preferred shape.

Ngôn ngữ chính
Go
Star
1.6k
Fork
266
Merge trung bình
3 ngày 6 giờ
Pull request đã merge (30 ngày)
22

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/Personal-AI-Router

Tất cả issue của NVIDIA/Personal-AI-Router

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.