Delivery watchdog: add run-liveness and decision-effect checks, not only rate alarms

Open Beginner friendly
  • Is something wrong?
#2,166 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Domain
documentation

Research direction

Find the watchdog guidance for agentic-delivery and agentic-ceo, then read how it currently describes rate alarms. Add guidance for run liveness and decision-effect checks, with one example of each, alongside the rate alarms. Done means the guidance covers both checks and includes both examples.

Written by the indexing model from the issue text.

Description

Finding: a conductor's delivery watchdog that measures only outcomes (merges per hour, ready-queue age) cannot tell a slow pipeline from a stopped one. A silently dead merge-queue run and a posted-but-unexecuted decision both read as "slow" for hours.

Observed (one multi-agent repo, one day):

  • A merge-queue run started with 2 PRs, posted no outcome, and no new run started for over 2 hours. The watchdog alarmed on the merge rate the whole time and never named the cause.
  • The conductor had decided to fill each run to 12 PRs. Over the next 24 hours, 35 runs carried a median of 2 to 3. No check compared the decision with the executor's next run.
  • 12 ready PRs rode 119 runs between them with no failure attributed to them, and never landed.

Proposed change (agentic-delivery / agentic-ceo):

  1. Run liveness. Alarm when the newest queue-run start is older than 1.5x the run's expected duration while ready work exists. The data is usually already present in the per-PR "queued in run X" markers.
  2. Decision effect check. When a conductor posts a decision to an executor, record its observable effect (for example "next run size >= 12") and assert it on the next watchdog cycle. A decision with no effect check decays silently.
  3. Rides without landing. Count each ready item's rides with no attributed failure. A high count points at dead or aborted runs, not at the item.

Done when: the watchdog guidance lists liveness and decision-effect checks next to the rate alarms, with one example each.

Dominant language
Python
Stars
2
Forks
1
Avg merge
1h 32m
Merged PRs (30d)
530

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from protocol/perun

All issues in protocol/perun

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.