fix(digest): relay-auth failure exits 0 — a dead digest cron reports green

Open Beginner friendly
#8,215 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
85/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript
Domain
backend, devops

Research direction

Start in scripts/seed-digest-notifications.mjs at the rules-fetch branches around lines 2183-2200, then inspect the existing brief-compose exit path for the intended telemetry flush behavior. Add coverage for relay authentication or network failures and for zero due rules, verifying that failures exit non-zero while zero-work runs exit 0.

Written by the indexing model from the issue text.

Description

Summary

When the digest cron cannot authenticate to the Convex relay, it exits 0. Railway records the run as a success, so a total delivery outage looks like a healthy green cron.

Observed during the #8208 outage: the 02:38Z run on 2026-09-16 fetched zero rules, sent zero digests, and completed "successfully".

2026-09-16T02:38:59Z [inf]  [digest] Cron run start: 2026-09-16T02:38:58.902Z
2026-09-16T02:39:03Z [inf]  [digest] watchlist scan: hashes=2293 candidates=76 events=0 enqueued=0
2026-09-16T02:39:04Z [err]  [digest] Failed to fetch rules: 401

That is the entire run. Nothing else fired.

Cause

scripts/seed-digest-notifications.mjs:2183-2191 — the non-ok branch logs, writes run meta, and returns from main(). main() resolves, the .catch at the bottom of the file never runs, and the process exits 0:

if (!res.ok) {
  console.error('[digest] Failed to fetch rules:', res.status);
  await writeDigestLastRunMeta({
    startedAtMs: nowMs,
    status: 'error',
    errorReason: `fetch_rules_http_${res.status}`,
  });
  return;
}

The catch arm below it (fetch_rules_failed:*, :2192-2200) has the same shape, so a network failure to Convex is equally silent.

Impact

The only thing that surfaced the outage was the digestNotifications staleness check in /api/health, which needs maxStaleMin 90 to elapse first. Compare notification-relay, which logged a 401 per event and stayed loud — that is the behaviour we want here.

This also mattered for triage: the diagnose-railway-seeders skill classifies on crash signatures, and a green run emits none.

Proposed fix

Treat "could not reach or authenticate to the relay" as a failed run: process.exit(1) after writeDigestLastRunMeta, matching the existing brief-compose gate at the end of main() (which already flushes telemetry before exiting). Both the !res.ok and the catch arm should do it.

Deliberately not in scope: the No digest rules found path (:2202) is a legitimate zero-work run and must stay green.

Acceptance

  • A run whose rules fetch 401s / 5xxs / times out exits non-zero and shows red in Railway.
  • A run with zero due rules still exits 0.
  • Test covering both, since the difference is one return vs process.exit(1) and nothing else catches it.

Context: #8208, and the validation comment https://github.com/koala73/worldmonitor/issues/8208#issuecomment-5691556926

Dominant language
TypeScript
Stars
87.1k
Forks
13.2k
Avg merge
8h 42m
Merged PRs (30d)
848

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from koala73/worldmonitor

All issues in koala73/worldmonitor

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.