Enable Over overflow when Cleo has no idle CI slot

Open
#569 1 comment 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@DeandreT is already working on this.

Since Oct 25, 2026.

Assessment

This issue has not been assessed yet.

Description

Allow Over to take automatic CI work when Cleo has no idle worker or a confirmed outage. This replaces the offline-only policy for Ferrule and Wareboxes.

Scope: scripts/ci/runner-failover.py, its existing test suite, install-runner-failover.sh service description and docs/ovh-ci.md. Keep this increment dependent on runner foundation #462 and separate from #458's headroom and #564's C# oracle corrections.

Policy: after two separated complete inventories (existing 20–120 second freshness window), enable the known Over runner's automatic/legacy labels if all expected Cleo workers are offline or every online Cleo worker is busy. Remove only those labels on the next poll when any Cleo worker is online and idle. Require boolean busy data; incomplete or ambiguous snapshots refuse decisions. Explicit host choices and unrelated labels survive. Running jobs remain untouched; credentials stay on Vecta.

Exclusions: no extra Cleo workers, CPU/memory/storage budget changes, workflow/test fixture reductions, forced outages, job cancellations/restarts or release deployment.

Acceptance:

  • Qualify fully busy, partly offline/busy, idle recovery, transient/stale observations, malformed busy data and unavailable backup decisions for both pools.
  • Preserve existing inventory, API failure, partial label mutation and durable-state controls.
  • Run focused controller and routing checks; publish a scoped dependent PR and require fresh full CI before merging.
  • Install the exact tested controller on Vecta, preserve previous source/state and verify live labels against fresh inventory without interrupting any worker.
  • Update the merge monitor to retain this authorized overflow policy.

Only overflow routing changes; Cleo worker capacity is assessed separately.

Implemented in #570 and installed on Vecta with previous source/state and worker/resource budgets preserved. Live inventories confirmed Over eligibility for busy Ferrule slots and no eligibility while Wareboxes Cleo was idle. The monitor retains this policy. Full #570 CI remains unqualified; fresh retargeted/main-integration qualification follows #462.

Dominant language
Rust
Stars
1
Forks
0
Avg merge
2h 50m
Merged PRs (30d)
571

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DeandreT/ferrule

All issues in DeandreT/ferrule

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.