[ZCode Desktop 3.16.3] Web Remote Control (relay V2): ~11 s connect-timeout retry loops and silent dropping of buffered outbound payloads (user messages lost)

Start by locating the web-remote-control code that emits the V2 connection retry and dropped buffered outbound payloads log messages; the issue does not identify source files or tests. Trace the retry decision and outbound buffer handling, then check how the connection is restored and whether dropped messages are surfaced or resent. Done means explaining the timeout and preventing silent message loss, or reporting an explicit error when delivery cannot be recovered.

[ZCode Desktop 3.16.3] Web Remote Control (relay V2): ~11 s connect-timeout retry loops and silent dropping of buffered outbound payloads (user messages lost)

Environment

• ZCode Desktop 3.16.3, macOS, arm64 (darwin 25.6.0). App updated to 3.16.3 on Oct 23 ~10:54 local; symptoms were observed on the previous version as well and continue on 3.16.3.
• Surface: webRemoteControlRelayV2 device session against zcode.z.ai (device connected to the relay; clients attach to it from the web/mobile).
• Remote target: a home server reached through the relay. The same symptoms were also seen when connecting over a direct Tailscale link (no relay), though that path still needs re-verification with 3.16.3 on the server side.
• Affected clients: both the desktop app and the mobile web client.

Symptom

Since ~Oct 22 the relay device connection degrades during active use. Reconnect cycles appear every 2–4 minutes; the worst observed gap between consecutive reconnect cycles is 26 s. From the user's point of view: responses can't be read and typed messages silently fail to send.

Log evidence

Source: ~/.zcode/v2/logs/YYYY-MM-DD.log, lines tagged web-remote-control.

1. Consistent failure signature. The first connect attempt stalls for a fixed ~11–12 s, times out (V2 connection retry), and the retry then succeeds within seconds. Example from Oct 23:

[2026-10-23 20:43:19.034] [web-remote-control] V2 device state {"state":"connecting"}
[2026-10-23 20:43:30.057] [warn] [web-remote-control] V2 connection retry        (+11.0 s)
[2026-10-23 20:43:38.150] [web-remote-control] V2 device token acquired          (+8.1 s)
[2026-10-23 20:43:38.526] [web-remote-control] V2 device state {"state":"authenticating"}
[2026-10-23 20:43:38.741] [web-remote-control] V2 device state {"state":"waiting_terminal"}

[2026-10-23 21:16:29.879] [web-remote-control] V2 device state {"state":"connecting"}
[2026-10-23 21:16:41.519] [warn] [web-remote-control] V2 connection retry        (+11.6 s)
[2026-10-23 21:16:49.488] [web-remote-control] V2 device token acquired          (+8.0 s)
...

The repeatable ~11 s stall before every retry looks like a fixed client-side connect timeout against a stalling path. No WebSocket close codes or reasons are logged, so the logs don't say why the attempt is abandoned.

2. Outbound payloads are silently dropped. User-visible message loss corresponds to these warnings:

[2026-10-24 09:58:30.474] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":2}
[2026-10-24 10:27:08.141] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":1}
[2026-10-24 10:33:50.503] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":1}
[2026-10-24 10:42:23.351] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":1}
[2026-10-24 11:21:58.084] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":1}

Note: the 10:27 / 10:33 / 10:42 drops happen without any logged reconnect cycle around them — no connecting / connection retry nearby, i.e. the transport degrades without a visible disconnect in the log.

3. Daily counts on this device (grep -c over the daily logs):

date	V2 connection retry	dropped buffered outbound payloads
2026-10-20	0	0
2026-10-21	0	0
2026-10-22	4	5
2026-10-23	21	4
2026-10-24 (as of ~11:30)	15	5

Sharp onset on Oct 22 after a clean Oct 20–21.

Correlated capture (VPN off): reconnect storm during measured path degradation

To rule the local VPN out completely, we ran a 15-minute continuous ICMP probe to the relay endpoint alongside the ZCode log, with the full-tunnel VPN disabled and routing verified direct (via the physical interface):

450 packets transmitted, 412 received, 8.4% packet loss
rtt min/avg/max/stddev = 21.9 / 363.4 / 134973.7 / 6639.9 ms
(first ~200 probes clean, loss burst afterwards; the max value is a reply arriving 135 s late)

In the same window the live WebSocket connection to the relay failed too:

[2026-10-24 11:19:20.169] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:19:22.469] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:19:26.639] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:19:34.709] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:19:49.858] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:20:15.412] [warn] [web-remote-control] V2 connection retry
[2026-10-24 11:20:54.704] [info] [web-remote-control] V2 device token acquired   (restored; ~90 s down)
[2026-10-24 11:21:58.084] [warn] [web-remote-control] dropped buffered outbound payloads {"windowId":1,"session":"d_T5…9U","droppedCount":1}

The ICMP loss burst started ~2 minutes before the retry storm. Cloud-fronted endpoints sometimes rate-limit ICMP, so the ping numbers alone would be ambiguous — but the simultaneous WebSocket failure shows the degradation hits the real relay path, not just ICMP. Note that short point-in-time probes (12/12 clean TLS handshakes, 10–30 ms) minutes earlier did not catch it; continuous monitoring was required.

Ruled out on the reporter's side

• VPN: reproduced with a WireGuard-based full-tunnel VPN both ON and OFF. Routing verified each time (direct path via the physical interface when OFF, no 0/1+128.0/1 hijack). See the correlated capture above for a measured degradation episode with the VPN off.
• Direct connection: reconnects and unsent messages also occurred over a direct Tailscale link to the server (relay not in the path) — to be re-verified after the server is updated to 3.16.3.
• Single client: both desktop (macOS) and mobile web clients are affected.

Timing correlation (may be coincidental)

Onset (~Oct 22) coincides with (a) public user-submitted outage reports for z.ai on Oct 22–23 and (b) the 3.16.2 / 3.16.3 releases, whose changelogs list several remote-connection fixes ("Fixed several remote connection compatibility issues", "Remote sessions now support persistent processes…", "Reduced message sending latency in remote sessions", "Fixed issues with the mobile remote terminal").

Questions

1. Is there a verbose/debug mode for web-remote-control that would log WebSocket close codes/reasons and the retry decision? The current log lines don't say why the connection is considered dead.
2. Is dropping buffered outbound payloads intended behavior? Losing typed user messages silently (no UI indication, nothing resent after reconnect) looks like a defect — the expected behavior would be flushing the buffer once the connection is re-established, or surfacing an explicit error.
3. Were there known relay-side incidents around Oct 22–24?
4. Which server-side logs should we collect from the remote Linux host to complete the picture? Happy to run additional diagnostics on request.