A single unresponsive/untrusted SSH host crashes unrelated plugins via unhandled "no valid actor" rejections, eventually disabling core plugins (including the UI layout) through the 5-crashes-in-10-minutes breaker
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 18/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- docker, typescript
Research direction
The failure starts in the host-metrics stats poller, which logs stats_connect_failed and stats_host_key_verification_failed for hosts 5 and 7-11. The unhandled rejection surfaces in the sandbox RPC layer, plugins/sandbox/rpc.js (fromWireError and Endpoint.dispatch), and the breaker lives in plugin_runtime. Start by reproducing with one unreachable host and checking whether a rejected stats promise reaches the sandbox dispatcher. Done means host-metrics errors stay inside that plugin and no unrelated plugin is restarted or disabled.
Written by the indexing model from the issue text.
Description
What happened?
Environment
- Termix version: 26.11.3 (
ghcr.io/termix-ssh/termix:latest) - Deployment: Docker
Note on scope
This is a different bug from the plugin-sandbox mass-restart issue previously reported and fixed. That fix is confirmed working in 26.11.3 — the supervisor now correctly logs "N of N plugins stopped answering at once; waiting instead of restarting them, since core or its channel to them was busy" instead of SIGKILLing the whole fleet simultaneously. This report is about a new failure mode that surfaced afterward.
Summary
A single host that's slow to respond over SSH, or has an untrusted/unverified host key, causes the host-metrics background stats poller to repeatedly fail against it. These failures appear to trigger unhandled promise rejections in unrelated plugins (observed in ssh-terminal: "This plugin call has no valid actor"), which crash those plugin processes. Because this happens repeatedly within a short window, several plugins individually hit Termix's "stopped 5 times in 10 minutes" circuit breaker and get permanently disabled (not just restarted) — including, eventually, the active UI layout plugin itself, taking down the entire web UI with no automatic recovery.
Evidence
host-metrics repeatedly failing against problem hosts
[3:36:29 PM] [WARN] [🖥️] Host has no saved key and nobody to ask [op:host_key_unverified,user:j2gj0GtMvFuXqXn6rLFwZ,host:7] [3:36:29 PM] [ERROR] [🧩] Stats collector host key verification failed {"operation":"stats_host_key_verification_failed","hostId":7}: This host's key is not trusted yet. Open a terminal to the host once to check and trust it. [op:plugin:host-metrics] ... [3:37:12 PM] [ERROR] [🧩] Stats collector connection failed {"operation":"stats_connect_failed","hostId":5}: SSH connection timeout [op:plugin:host-metrics] ... [3:49:47 PM] [ERROR] [🧩] Stats collector connection failed {"operation":"stats_connect_failed","hostId":5}: SSH connection timeout [op:plugin:host-metrics]
Host 5 (slow/unreachable over SSH) and hosts 7–11 (untrusted keys) produced these errors repeatedly over the ~15 minute window captured.
Unrelated plugin throwing an unhandled rejection shortly after
[3:43:20 PM] [WARN] [🧩] Plugin ssh-terminal: Unhandled rejection: Error: This plugin call has no valid actor [op:plugin_sandbox] [3:43:20 PM] [WARN] [🧩] Plugin ssh-terminal: at fromWireError (file:///app/dist/backend/backend/plugins/sandbox/rpc.js:606:19) [op:plugin_sandbox] [3:43:20 PM] [WARN] [🧩] Plugin ssh-terminal: at Endpoint.dispatch (file:///app/dist/backend/backend/plugins/sandbox/rpc.js:501:35) [op:plugin_sandbox]
Individual plugins crash-looping independently (not a mass-restart event)
[3:48:09 PM] [ERROR] [🧩] Plugin vault stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL) [3:48:09 PM] [ERROR] [🧩] Plugin alerts stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL) [3:51:06 PM] [ERROR] [🧩] Plugin totp stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL) [3:51:06 PM] [ERROR] [🧩] Plugin webauthn stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL) [3:51:06 PM] [ERROR] [🧩] Plugin ssh-terminal stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL)
Shortly after, the active layout plugin itself tripped the same breaker, taking down the whole UI (screenshot available on request — shows Termix's own "This layout could not be shown" recovery screen, with the same "stopped 5 times in 10 minutes" message).
Expected behavior
A background polling failure against one slow or not-yet-trusted host should be caught and handled entirely within host-metrics (e.g., logged, skipped, backed off) and must never be able to crash unrelated plugin processes — especially not ssh-terminal, authentication plugins (totp, webauthn), or the active UI layout plugin. None of these have any direct relationship to a single host's SSH reachability.
Actual behavior
A single flaky/untrusted host caused a cascade of unrelated individual plugin crashes, several of which tripped the 10-minute circuit breaker and were permanently disabled (no automatic recovery) — including, eventually, the UI layout plugin, which took the entire web interface down with no path back except Termix's manual "Start Classic Layout again" recovery button, plus manually re-enabling the other disabled plugins (ssh-terminal, totp, webauthn, vault, alerts) from Settings → Plugins.
Workaround
Removing the problem host (ID 5) from the host list stopped the recurrence. This only removes the trigger, not the underlying issue: any future flaky/unreachable/untrusted host could reproduce this and take down the UI again.
Impact
A routine, expected condition (one slow or newly-added host) was able to cause a full outage of the web UI requiring multiple manual recovery steps, with no self-healing.
Steps to reproduce
- Add a host that is slow/unreachable over SSH (or whose SSH host key has never been trusted via an interactive terminal session).
- Leave Termix running with
host-metricsenabled so it periodically polls that host in the background. - Watch
docker logs -f <container>— within a few minutes, unrelated plugins (not justhost-metrics) start loggingstopped answering, restarting it/exited (SIGKILL)individually, and eventually:
[ERROR] [🧩] Plugin stopped [op:plugin_runtime] Error: The plugin process stopped 5 times in 10 minutes: exited (SIGKILL) - If the active layout plugin (
classic, formerlyclassic-layout) is among the plugins that trip this breaker, the entire UI goes down with:
This layout could not be shown — The layout failed to start.
classic | Classic Layout 1.2.0
The plugin process stopped 5 times in 10 minutes: exited (SIGKILL)
What did you expect?
No response
Termix version
26.11.3
Platform
Web browser
Server install
Docker
Environment
No response
Logs
Anything else?
No response
Before you submit
- I searched open and closed issues and this is not a duplicate
- I am on the latest version, or I said which version I am on
- Dominant language
- TypeScript
- Stars
- 16.1k
- Forks
- 776
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 57
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Termix-SSH/Termix
-
bug platform-desktop platform-windows
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Termix-SSH/Termix#1951 · 2 comments ·
Maintainers usually reply within 1 day
-
feature
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Termix-SSH/Termix#1843 · 1 reaction ·
Maintainers usually reply within 1 day
-
bug platform-android
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Termix-SSH/Termix#1829 · 2 comments ·
Maintainers usually reply within 1 day
-
bug platform-windows
Difficulty 3/5 1-2 days Newbie friendliness 52/100
Termix-SSH/Termix#1953 · 1 comment ·
Maintainers usually reply within 1 day
-
bug platform-docker platform-web
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
Termix-SSH/Termix#1950 · 1 comment ·
Maintainers usually reply within 1 day
All issues in Termix-SSH/Termix
Similar issues
-
clawsweeper:needs-maintainer-review clawsweeper:needs-product-decision clawsweeper:no-new-fix-pr clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster P3
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
openclaw/openclaw#203385 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
nwverse/persona#12800 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Maintainers usually reply within 1 day
-
[MCP]: page snapshots in .playwright-mcp/ are not git-ignored (.playwright-cli/ is, since #42318)Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
microsoft/playwright#44289 ·
Maintainers usually reply within 1 day
-
good first issue
Difficulty 2/5 Half a day Newbie friendliness 65/100
Maintainers usually reply within 1 day