Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

A gateway sends chosen fleet events to a signed generic webhook

Aperta
#719 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
node.js, typescript
Ambito
api, backend

Direzione di ricerca

Start with the named entry points in src/core/config.ts, src/contract/schemas.ts, and src/daemon/main.ts, then read the referenced architecture rules and existing gateway tests. The scope also calls for a new alerts module, a webhook port and test fake, protocol version updates, dependency rules, and documentation. Done means configured generic destinations send signed, ordered events with bounded retries without blocking the bus, while secrets and URLs are redacted and workers never send.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

task:ready

Part of #707.

Scope

A gateway reads gateway.alerts.destinations and sends each chosen event to every generic destination that lists it, as signed JSON. Sending is bounded, in order per destination, and never holds up the bus. Webhook URLs and secrets are masked in the log, the daemon.started event and the event history. After this PR, an operator can point n8n, Zapier or their own receiver at the gateway. slack comes in the next task; format takes only "generic" here.

In short

A gateway with a generic destination POSTs each listed event to its URL, with a signature the receiver can check. A receiver that is down gets three tries, then a warning in daemon.log. A worker never sends.

flowchart LR
  E[Fact on the gateway's bus] --> Q["alerts: destination queue (new)"]
  Q --> P["POST, signed (new)"]
  P -->|2xx| D[delivered]
  P -->|"network error, timeout, 429, 5xx"| R["wait 1 s, then 5 s; 3 tries in all"]
  P -->|other status| G[gave up: warn]
  R --> P
  • try: one POST of one message; 10 s at most, counted from the POST's start. A message gets at most 3 tries.
  • At most 100 messages wait per destination, not counting the one being sent.

Config, before and after:

$ simlock config set gateway.alerts.destinations '[{"name":"audit","format":"generic","url":"http://n8n.internal/hook","secret":"s","events":["worker.connected"]}]'
# before
{"error":{"code":"USAGE","message":"Invalid config after setting gateway.alerts.destinations: Unknown config key: \"gateway.alerts\""}}
# after
{"error":{"code":"USAGE","message":"Invalid config after setting gateway.alerts.destinations: Invalid config value for \"gateway.alerts.destinations[0].url\": expected an https:// URL, or an http:// URL to localhost, 127.0.0.1 or [::1], with no user name or password"}}

Inputs (each is one destination; every other field valid):

Field Input After
name audit, ops-1 accepted
name Audit, 1ops, a_b, "", 33 characters, missing expected a name of lowercase letters, digits and dashes that starts with a letter, at most 32 characters
name the same as an earlier destination's expected a name no other destination uses
format generic accepted
format slack, json, missing expected one of "generic"
url https://n8n.example.com/x, http://localhost:5678/x, http://127.0.0.1/x, http://[::1]:80/x accepted
url http://n8n.internal/x, ftp://example.com, n8n.example.com/x, https://user:pw@host/x, https://user@host/x, "", missing expected an https:// URL, or an http:// URL to localhost, 127.0.0.1 or [::1], with no user name or password
events ["worker.connected"] accepted
events [], ["lease.granted"], ["worker.connected","worker.connected"], missing expected a non-empty list of alert events, each once: worker.connected, worker.disconnected, worker.rejected, worker.drain-started, worker.drain-ended, device.quarantine-abandoned, device.quarantine-stranded, device.recovery-failed, device.stalled-transition-detected, disk.pressure-detected, component.install-failed
secret "s3cr3t" accepted
secret "", missing expected a non-empty secret for a generic destination
the list 17 destinations, an object, a string expected a list of at most 16 destinations
one entry "x", null, [] expected an object
one entry an extra key, e.g. "headers": {} warning Unknown config key: "gateway.alerts.destinations[0].headers"; the gateway starts; simlock config set refuses it (as every unknown key today)

When several fields are bad, the first in the order name, format, url, events, secret is reported, and the first bad destination in the list.

Each message is Invalid config value for "<path>": expected <what>, with <path> like gateway.alerts.destinations[2].url. A gateway and a worker both refuse a bad block at start. A worker sends nothing whatever the block says.

If this goes wrong, a receiver gets nothing, gets events out of order, or a dead receiver slows the gateway.

Technical spec

Facts

Commit ccb50353.

# Fact Source
F1 EventBus.subscribe(event, handler) returns an unsubscribe function; the handler is synchronous and returns void. src/bus/index.ts:416, :493
F2 Subscribers run synchronously on emit; a synchronous throw is logged ("Event handler failed") and skipped. src/bus/index.ts:483, :546-552
F3 A worker's events reach the gateway's bus through republish, with workerId added to the payload and the worker's id and timestamp kept. src/gateway/worker-link.ts:533, :541
F4 worker.connected, worker.disconnected, worker.rejected, worker.drain-started, worker.drain-ended are emitted by the gateway itself. src/gateway/worker-registry.ts:251, :334, :444, :474; src/bus/index.ts:290-292
F5 The envelope is id, seq, timestamp, event, payload, module. src/contract/schemas.ts:777-786; docs/EVENTS.md:183
F6 objectValidator warns "Unknown config key" for a key not in its shape and keeps going; invalidValue(path, expected) builds Invalid config value for "<path>": expected <expected>. src/core/validation.ts:17-40
F7 The gateway block's validator lists url, token, label, three timeouts and routing; it runs in both modes. src/core/config.ts:858-866, :381-430
F8 No config key is a list of objects today; stringArray is the only list validator. src/core/validation.ts:133-142; search Array.isArray in src/core/config.ts
F9 redactConfig masks only gateway.token with REDACTED_VALUE (returning its input unchanged when the token is unset), and feeds the daemon.started payload and the "Daemon started" log line. src/core/config.ts:345-347; src/daemon/server.ts:543-558
F10 config.get answers the config as loaded, admin role. src/daemon/dispatcher.ts:788; src/contract/operations.ts:614-618
F11 config.get's output schema lists every gateway key; zod strips a key it does not list, and config.test.ts compares the loaded key set with the schema's. src/contract/schemas.ts:937-958; src/core/config.test.ts:1902
F12 simlock config set parses its value with JSON.parse and re-runs loadConfig, turning an "Unknown config key" warning into an error. src/cli/commands/config.ts:88-94; src/cli/environment.ts:200-214
F13 The gateway branch of main.ts is startGatewayDaemon; its dispose stops the fleet coordinator, owner-routed facts and the gateway service. src/daemon/main.ts:208-215, :699, :888-892
F14 startDaemon takes test seams as options, e.g. hostLoadProbe?: HostLoadProbe | null, and configOverrides. src/daemon/main.ts:114-130; src/daemon/gateway-fleet.e2e.test.ts:128-139
F15 Clock has now, setTimer, cancel; FakeClock serves tests. src/ports/clock.ts:5-9, :31
F16 No code sends HTTP to a remote host today; architecture rule: every external API goes through a port. search fetch(, https.request, undici in src/; docs/internal/agent-rules/architecture.md:40
F17 Every contract shape change bumps PROTOCOL_VERSION_RANGE by one. src/contract/AGENTS.md:59; src/contract/protocol.ts:13
F18 gateway-fleet.e2e.test.ts starts a real gateway (startGateway, line 124) and real workers (startWorker, line 176) over a loopback uplink. src/daemon/gateway-fleet.e2e.test.ts:124-176
F19 dependency-cruiser rules are direction(name, comment, from, to) entries; host-load has an "imports ports only" rule and a "nobody imports it" rule. .dependency-cruiser.mjs:130-141
F20 docs-manuals.test.ts requires an AGENTS.md and a CLAUDE.md that only imports it in every module, and a row in docs/internal/ARCHITECTURE.md. src/docs-manuals.test.ts:64-80; docs/internal/ARCHITECTURE.md:57
F21 WHATWG new URL("http://[::1]:80/x").hostname is [::1]; new URL("n8n.example.com/x") throws. Node 24 REPL
F22 Node's fetch (undici) with redirect: "manual" resolves a 3xx with its real status and does not follow it; plain WHATWG Fetch would give status 0. Node 24.14.0: a loopback 301 resolved status 301
F23 Node's fetch rejects a URL with a user name or password, and its error message holds the whole URL. A network failure rejects with TypeError: fetch failed whose cause.code is e.g. ECONNREFUSED, ENOTFOUND. Node 24.14.0: fetch("https://user:[email protected]/hook") → "Request cannot be constructed from a URL that includes credentials: https://user:[email protected]/hook"
F24 config.test.ts's key-set walk treats an array as a leaf and loads the default config. src/core/config.test.ts:38-41, :1914-1931
F25 dependency-cruiser runs in pnpm lint, which the commit hook and CI run; agents do not. package.json:58; docs/internal/agent-rules/toolchain.md:20-27
F26 FakeDriver.failOn("installComponent", …) makes an install fail, and the worker emits component.install-failed. src/core/fake-driver.ts:334-345
F27 On stop, stopAuxiliary (module drain, HTTP gateway stop, which destroys every accepted socket, the uplinks included) runs before dispose. src/daemon/server.ts:610, :624; src/http/server.ts:207-219; src/daemon/main.ts:893-897
F28 ROUTING_POLICIES lives in core and the gateway's registry is typed against it, so a mismatch fails typecheck in main.ts. src/core/config.ts:82-85; src/daemon/main.ts:733-736
Modules touched

Searches: redactConfig, GATEWAY_CONFIG_KEYS, configSchema, startGatewayDaemon, REDACTED_VALUE, ARCHITECTURE.md module table.

  • src/core/config.ts: ALERT_EVENTS (as const), AlertEventName, ALERT_FORMATS (["generic"]), AlertFormatName, ALERT_FORMAT_SECRET, AlertDestination type, gateway.alerts.destinations in Config, its default [], the validator, and redactConfig masking url and secret.
  • src/core/validation.ts: a list-of-objects helper if the validator needs one.
  • src/contract/schemas.ts: gateway.alerts.destinations in configSchema: name, format: z.enum(["generic"]), url, events: z.array(z.string()), secret: z.string().optional() (optional so #720's slack destinations parse). events stays a plain string list: the contract imports nothing from core, and a second copy of the eleven names would need a test that the two agree. PROTOCOL_VERSION_RANGE and every test that pins the current number (search the number itself and "protocol " in src/, e.g. src/contract/protocol.test.ts, src/daemon/server.test.ts, src/daemon-protocol/index.test.ts, src/gateway/service.test.ts): bump by one, min with it.
  • src/ports/webhook-poster.ts (new): WebhookPoster, FetchWebhookPoster, FakeWebhookPoster (in src/ports/testing.ts). Exported from src/ports/index.ts.
  • src/alerts/ (new): index.ts (createAlerts), the queue, formats/ (types, registry, generic/, conformance suite), AGENTS.md, CLAUDE.md.
  • src/daemon/main.ts: startGatewayDaemon builds it when the list is not empty, before daemon.start() (whose converge opens the uplink listener, so no worker can join before alerts subscribes). It stops alerts first in stopAuxiliary, before the HTTP gateway closes the worker uplinks, so a gateway's own stop sends no worker.disconnected (F27). StartDaemonOptions.webhookPoster?: WebhookPoster.
  • .dependency-cruiser.mjs: alerts-imports-bus-ports-core-only and not-alerts (only src/daemon/main.ts imports it); alerts joins no-fs-or-process.
  • Docs: docs/CONFIGURATION.md (a gateway.alerts.destinations row and an "Alerts" section), docs/internal/ARCHITECTURE.md (module row, the two rules), src/daemon/AGENTS.md (gateway composition line).
flowchart LR
  main["daemon/main.ts *"] --> AL["alerts/ (new)"]
  main --> CFG["core/config.ts *"]
  AL --> BUS[bus]
  AL --> WP["ports/webhook-poster.ts (new)"]
  AL --> CFG
  CS["contract/schemas.ts *"]
sequenceDiagram
  participant W as Worker
  participant G as Gateway bus
  participant A as alerts
  participant R as Receiver
  W->>G: device.quarantine-abandoned (republished, workerId)
  G->>A: handler: enqueue, return
  A->>R: POST (signed), try 1
  R-->>A: 503
  A->>R: try 2 after 1 s
  R-->>A: no answer in 10 s (abort)
  A->>R: try 3 after 5 s
  R-->>A: 503
  A->>A: warn "Alert send gave up"
Surface and extension points
// src/alerts/index.ts
export function createAlerts(options: {
  destinations: readonly AlertDestination[];
  eventBus: EventBus;
  poster: WebhookPoster;
  clock: Clock;
  logger: Logger;      // main passes logger.child("alerts")
  version: string;     // for User-Agent
}): Alerts;

export interface Alerts {
  stop(): void;
}

createAlerts subscribes at once: one subscribe per event name that at least one destination lists (F1).

Formats are an extension point: #707 names two (generic here, slack in #720), so architecture rules 19-21 apply.

// src/alerts/formats/types.ts
export interface AlertFormatDefinition {
  readonly name: AlertFormatName;            // from core's ALERT_FORMATS
  build(input: {
    envelope: EventEnvelope; destination: AlertDestination;
    version: string; workerLabel(id: string): string;
  }): { headers: Record<string, string>; body: string };
}
// src/alerts/formats/index.ts
export const ALERT_FORMAT_REGISTRY: { readonly [N in AlertFormatName]: AlertFormatDefinition };
  • One directory per format: src/alerts/formats/generic/ here.
  • The queue looks a destination's format up in the registry and calls build; it never compares against a format name (rule 21).
  • Core cannot import alerts, so the config's names live in core as ALERT_FORMATS (["generic"] here), as ROUTING_POLICIES does for routing (src/core/config.ts:82). The registry's mapped type over AlertFormatName makes a missing or extra format a type error (rule 10). Which formats need a secret is a core table beside it, ALERT_FORMAT_SECRET: Record<AlertFormatName, "required" | "refused">, because the check runs at config load.
  • AlertFormatDefinition is {name, build}, not rule 20's {name, config schema, create}: a format's only config rule (its secret) is checked by core at load, which cannot import alerts, and build holds no state, so there is nothing to create.
  • A conformance suite (describe.each over the registry) checks every format: build returns a JSON body, Content-Type: application/json and User-Agent: simlock/<version>, and never throws on the eleven events.
Contract and event changes
  • config.get output: gateway.alerts: { destinations: AlertDestination[] }. This PR bumps the protocol by one, min with it (F17). Other open tasks bump the same line (today #713, #714, #629-#632, #635-#637, #720, #721): the PR that merges second takes the base branch and bumps again.
  • No new event. A failed send is not an event (ADR 0035 §4).
  • Config fields, read from config.json and --config overrides, bounds as in the In short table. name ^[a-z][a-z0-9-]{0,31}$. url parsed by new URL; no username or password (F23); https: any host; http: only for hostname localhost, 127.0.0.1 or [::1] (F21). secret any non-empty string. The list at most 16 entries.
Other code on the same state

Searches: redactConfig (3 callers), config.gateway (12 hits), subscribeAll (2), subscribe( in src/gateway (4).

  • EventHistory and OwnerRoutedFactBus subscribe to the same bus. Subscribers are independent; order between them is not promised (docs/internal/agent-rules/events.md:20). The alerts handler only enqueues.
  • redactConfig masks url on every destination and secret only where one is set, and returns a copy whenever it masks anything.
  • redactConfig callers (server.ts:543, :558) get the extra masking with no change.
  • config.get (F10) keeps returning the URL and secret to admins, as for gateway.token.
  • WorkerLink republishes; alerts sees a relayed event once, as the bus delivers it.
Failure modes
Call Outcome What happens Limit, counted from
poster.post 2xx debug "Alert delivered"; next message —
poster.post 429 or 5xx wait, try again; after try 3 warn "Alert send gave up" with status waits 1 s, 5 s, from the try's end
poster.post 1xx, 3xx, 4xx but 429 warn "Alert send gave up" at once, with status; redirect not followed —
poster.post rejects (DNS, refused, TLS) as 5xx, with error = the rejection's cause.code (e.g. ECONNREFUSED), else network-error; never the error's message, which can hold the URL (F23) as above
poster.post no answer abort the signal and move on at once, without waiting for post to settle; as 5xx with error timeout. A late settle of that try is ignored 10 s from the POST's start
poster.post throws synchronously treated as a rejection; the bus never sees it —
enqueue 100 already waiting drop the oldest waiting, warn "Alert dropped: queue full". The message in flight, including one waiting between tries, is not one of the 100 —
stop() any later settle of an aborted try ignored: no log line after "Alerts discarded at stop" —
stop() message in flight (being tried, or waiting between tries) counted in the discarded count
stop() send in flight abort it, cancel timers, empty queues, unsubscribe; info "Alerts discarded at stop" with the count per destination that had any none: waits for nothing
  • State: queues are memory only. A restart loses them (accepted, ADR 0035 §4).
  • Building the request: once, when the message leaves the queue for its first try, never in the bus handler. Later tries resend the same headers and body. A throw while building it ends that message as "gave up" with error build-failed.
  • Two events at once: the bus is synchronous, so they enqueue in emit order; one send per destination at a time keeps it.
Rules in play
  • architecture rule 1, "Every external API is accessed through a port interface": WebhookPoster.
  • architecture, bounded cross-process waits, "Every wait that crosses a process boundary is bounded": 10 s per try, 3 tries, stop waits for nothing.
  • architecture, event bus for observers only: the handler enqueues and returns.
  • architecture rule 15, a module left out leaves the rest as before: no destinations, no module.
  • events rule 4–5, emitters fire and forget, a throwing subscriber is skipped: the handler never throws.
  • events rule 6, "public contract: additive changes only": the generic body is the existing envelope.
  • safety, "wire input is a claim": a receiver's reply is read only for its status; the body is cancelled unread.
  • safety, root validation fails closed: a bad destination stops the start.
  • ADR 0035 §1–§6.
Tests

Seam: createAlerts with a real EventBus, FakeClock and FakeWebhookPoster, in src/alerts/alerts.test.ts, modelled on src/gateway/owner-routed-facts.test.ts; config through loadConfig in src/core/config.test.ts; one case in src/daemon/gateway-fleet.e2e.test.ts.

  • posts a listed event to a generic destination as the envelope with Content-Type, User-Agent, X-Simlock-Event, X-Simlock-Delivery and a signature that verifies with the secret, and logs Alert delivered at debug (cases: a gateway-own worker.connected; a republished device.quarantine-abandoned whose body keeps workerId)
  • posts nothing for an event the destination did not list (cases: worker.disconnected not listed; lease.granted)
  • posts to every destination that lists the event, each signed with its own secret (two destinations)
  • tries again after a network error, a timeout, 429 and 5xx, 1 s then 5 s after the failed try, and gives up with a warn line after the third (one case per kind; the warn line has destination, event, eventId, attempt: 3 and status, or error ECONNREFUSED, network-error (no cause.code) or timeout)
  • gives up at once on a status that is not 2xx, 429 or 5xx, and posts it once (cases 301, 400, 404)
  • aborts a try with no answer at 10 s, not before, and moves on without waiting for it to settle (signal not aborted at 9 999 ms, aborted at 10 000 ms; a poster that never settles; a late settle after the retry starts is ignored)
  • sends one message at a time per destination, in the order the bus emitted them, the second only after the first's last try
  • a destination that does not answer does not delay another
  • keeps at most 100 waiting per destination and drops the oldest waiting one, never the one in flight, with a warn line
  • stop aborts the try in flight, posts nothing more, ignores later events and late settles, and logs the discarded count per destination as its last line
  • the bus handler returns before the post settles, and a poster that throws reaches neither the emitter nor other subscribers
  • a request that throws while being built ends as gave up with build-failed and the next message is sent
  • a message is built once when it leaves the queue, and later tries resend the same headers and body (a format whose output changes per call: try 2 and try 3 equal try 1)
  • config.get's format enum lists exactly core's ALERT_FORMATS (src/contract imports nothing from core, so a test checks the hand-written copy, rule 10)
  • no log line holds a destination's url or secret (over every line the tests above produce)
  • loadConfig accepts a valid generic destination and defaults gateway.alerts.destinations to an empty list
  • loadConfig refuses each bad destination field with its message (one case per refused input of the In short table, credentials included, on a gateway; one on a worker; one with two bad fields reporting name)
  • loadConfig warns about an unknown key inside a destination and still loads
  • redactConfig masks every destination's url and secret, keeps name, format and events, and leaves its input untouched (extends describe("redactConfig"), config.test.ts:1859; cases: token set and unset)
  • config.get's schema keeps every field of a loaded destination (a config with one destination, parsed through configSchema, deep-equals the loaded gateway.alerts; F24 says the key-set test cannot see this)
  • FetchWebhookPoster returns a 301 without following it, cancels the response body unread, and rejects when the signal aborts (src/ports/webhook-poster.test.ts, loopback server that sees its response stream closed)
  • every registered format passes the conformance suite (describe.each over ALERT_FORMAT_REGISTRY)
  • a refused connection through FetchWebhookPoster logs ECONNREFUSED and never the URL (createAlerts with the real poster and a closed loopback port; the URL carries a path token)
  • a worker with a destination posts nothing, while its gateway posts the relayed event (gateway-fleet.e2e.test.ts: the worker and the gateway each list component.install-failed with their own FakeWebhookPoster; after the gateway's history holds the worker's worker.connected, a component.install on the worker fails through FakeDriver.failOn (F26); the gateway's poster gets it with workerId, the worker's gets nothing)
  • stopping a gateway sends no worker.disconnected and aborts a try in flight (gateway-fleet.e2e.test.ts, two destinations: quick lists worker.disconnected and its poster answers 200 at once; stuck lists worker.connected and its poster never settles. A worker joins; daemon stop: quick got no post, so alerts stopped before the uplinks closed; stuck's signal is aborted, so main.ts called stop())

No test for "no destinations builds no module": with no destination there is nothing that could be posted, so no test could fail (testing rule 2). Building before daemon.start() has no test either: a worker cannot join before the listener opens inside start(), so a later build loses nothing a test can make happen; the order is reviewed in main.ts.

  • a worker joining a gateway with a generic destination makes a loopback receiver get one signed worker.connected naming that worker (gateway-fleet.e2e.test.ts, real FetchWebhookPoster)

Done when

  • CI is green: the lines above pass, and dependency-cruiser (run by pnpm lint in the commit hook and CI, F25) passes with the two new rules.
  • On a gateway started by gateway-fleet.e2e.test.ts with a generic destination, a loopback receiver gets worker.connected with a signature that checks out.
  • simlock config set gateway.alerts.destinations '[{"name":"audit","format":"generic","url":"http://n8n.internal/hook","secret":"s","events":["worker.connected"]}]' prints the URL error from In short and exits 2 (run with the built CLI and a scratch SIMLOCK_HOME).
  • docs/CONFIGURATION.md documents every field, the URL rule, the eleven events, the request headers, how to check the signature, and the retry rule. It names the event history as a whole, not events.jsonl, so it stays true when #629 lands.

Out of scope

  • format: "slack" (#720), simlock alerts test (#721).
  • Status or console display of failures.
  • Any event beyond the eleven.

Depends on

  • #718 (ADR 0035)

Written by an agent.

Lingua principale
TypeScript
Stelle
20
Fork
2
Merge medio
8h 17m
PR unite (30g)
273

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di callstackincubator/simlock

Tutte le issue di callstackincubator/simlock

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.