Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Cluster peer stuck offline when paired over non-multicast link (Tailscale / VPN) — mDNS does not traverse tailscale0

Đang mở
  • Có gì không ổn?
#91 1 bình luận 1 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 5 ngày

@ckelseynv đang làm issue này rồi.

Từ ngày 18/9/2026.

Đánh giá

Issue này chưa được đánh giá.

Mô tả

Cluster member stuck "offline" when paired over a non-multicast link (Tailscale / VPN)

Summary

After a successful PIN pairing between two nodes connected only via a non-multicast
link (Tailscale, ZeroTier, Hamachi, corporate VPN), the peer is permanently shown
as "offline" in the Overview tab and members.json lastSeen stays null,
even though every cluster TCP port (14318, 14319, 14321) is reachable both ways
and the broker log shows nvpair-ui-broker and nvpair-cluster-manager running.

Reproduction

  1. Two machines on the same Tailscale network (e.g. 100.100.108.73 and
    100.121.35.74), neither on the same LAN.
  2. Settings → Cluster → Add node → invite by IP (Tailscale IP) → accept PIN.
    Pairing completes; both members show state: "member".
  3. Engine-manager logs repeat forever:
    remote engine status for <peer-uuid> unavailable: -32000: node <peer-uuid> is not a discovered ec peer.
  4. members.json for both members shows lastSeen: null, even after minutes.
  5. The UI Overview tab never reports the peer as online.

Root cause

Presence / "online" status is driven by the mDNS discovery layer
(nvpair-node-scanner listening for _nvpair-node._tcp announcements), not by
the cluster-manager's roster probe. Multicast DNS (224.0.0.251) does not
traverse Tailscale's userspace WireGuard tunnels — both nodes send mDNS queries
on tailscale0 (mdns send: ... iface=tailscale0 ip=100.100.108.73) but no
peers' announcements ever arrive, so the peer never enters the discovery store.
Once absent from discovery, nvpair-engine-manager rejects every dial with
-32000: not a discovered ec peer, and nvpair-cluster-manager's lastSeen
timestamp is never updated.

Workaround that works today

Seed configs/manual-nodes.json (Electron-side persistence in
desktop/src/electron/service-bridge/manual-nodes-store.ts) and restart the app.
The supervisor's replayManualNodes() re-adds the entry via node/add over the
broker; nvpair-manual-nodes probes the peer directly over Tailscale every
~10 s, emits node/discovered, the broker merges it into the discovery
snapshot, and engine-manager stops rejecting the dial.

[
  {
    "id": "dabeast",
    "address": "100.121.35.74",
    "name": "dabeast"
  }
]

After app restart, logs confirm:

  • [nvpair-manual-nodes] INFO manual node state changed node_id=dabeast addr=100.121.35.74 ollama_up=true node_info_up=true models=13 gpus=2
  • discovery:nodes-changed carries the peer with full model inventory
  • Overview tab flips the peer to online

Suggested fixes (any one would address the root cause)

  1. Document the workaround. Surface manual-node entry in the desktop UI
    (currently Settings → Cluster → Add node does PIN-pairing only — see
    desktop/src/ui/components/AddNodeModal.tsx — there is no manual-node
    registration UI).
  2. Treat manual-nodes entries as authoritative for presence. When a peer
    was paired by IP and not seen via mDNS, fall back to nvpair-manual-nodes
    status for heartbeat / lastSeen updates.
  3. Add a Tailscale-aware discovery path. Either (a) a Tailscale MagicDNS /
    unicast DNS-SD fallback when mDNS browse times out, or (b) a peer relay
    over the cluster's existing mTLS channel so each node advertises its
    cluster peers to the others.
  4. Avoid the crash-on-add churn. Adding a manual node today restarts
    nvpair-cluster-manager; combined with the worker's in-memory state loss,
    this makes the pairing flow itself brittle.

Logs

14:23:05 [nvpair-manual-nodes] INFO log level changed level=debug
14:39:57 [nvpair-manual-nodes] INFO using stdio transport
14:40:30 [nvpair-ui-broker] ERROR nvpair-errors crashed; service-error pipeline
       unavailable until it restarts attempt=3
14:40:32 [nvpair-ui-broker] WARN supervisor: worker exited unexpectedly
       worker=cluster-manager attempt=3
14:40:42 [nvpair-engine-manager] WARN remote engine status for
       18e94d12-e71f-42c9-8c07-355f68bb2fb8 unavailable: -32000: node
       18e94d12-e71f-42c9-8c07-355f68bb2fb8 is not a discovered ec peer

(Repeats every ~20 s. Discovery nodes-changed lists only linuxbeast.)

Environment

  • nvpair 0.1.1 (pacman build from release/0.1.1/linux/NVPAIR-Setup-0.1.1-x64.pacman)
  • Linux 7.2.5-3-omarchy, kernel 7.2.5-3-omarchy
  • Tailscale 100.x on both nodes, direct peer-to-peer connection (active; direct)
  • 13 models on peer, 2 GPUs
Ngôn ngữ chính
Go
Star
1.6k
Fork
266
Merge trung bình
3 ngày 15 giờ
Pull request đã merge (30 ngày)
23

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/Personal-AI-Router

Tất cả issue của NVIDIA/Personal-AI-Router

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.