A gateway updates a chosen worker, incompatible ones included, and routes around it meanwhile
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
Research direction
Start with src/gateway/dispatcher.test.ts and the drain tests at lines 322–331, then trace worker.drain through src/gateway/dispatcher.ts, src/gateway/worker-registry.ts, and src/gateway/routing/stages/takes-requests.ts. Read the contract and CLI entry points listed in the scope, then run pnpm test:changed. Done means the specified worker and gateway update flows, routing and drain behavior, CLI output, HTTP routes, and docs are covered and CI passes.
Written by the indexing model from the issue text.
Description
Part of #825.
Scope
From a gateway, an operator updates one chosen worker, including one that shows incompatible. The ways in are simlock worker update <id> [--version X] [--force] [--detach], POST /v1/workers/{id}/update, and the client's updateWorker. The gateway routes no leases to an updating worker and never touches the drain set. The worker list shows each worker's version, its latest version and its update stage. The gateway updates itself with simlock update --gateway, with no lease wait.
Shared values: worker.update, the worker view fields, HTTP routes, WORKER_NOT_CONNECTED, the routing reason updating, the "newer" rule (read), events (read), the protocol version.
In short
simlock worker update 3f81a2c4 follows the update on that worker. It ends at mac-studio-2 (3f81a2c4) is running 0.4.0, or at Update failed (<stage>): <message>. Meanwhile, simlock worker list adds -- simlock 0.3.0 (0.4.0 available) -- updating to 0.4.0: waiting for 2 leases to that worker's line.
sequenceDiagram
participant O as simlock worker update
participant G as gateway
participant W as worker (old)
O->>G: worker.update {workerId}
alt disconnected
G-->>O: WORKER_NOT_CONNECTED
else connected or incompatible
G->>W: daemon.update (frozen; passes the degraded client)
W-->>G: { started, update: waiting }
G->>G: view.update set, routing skips it
G-->>O: answer
loop 30 s tick, and on daemon.update-* events
G->>W: status.get, or daemon.update {query} when incompatible
end
Note over O: O polls worker.list every 2 s
Note over G: worker restarts and dials back
G-->>O: view version 0.4.0, update null: running 0.4.0
end
| Input | Today | After |
|---|---|---|
simlock worker update 3f81a2c4 |
Unknown worker command: update |
follows to … is running <v> |
simlock worker update nope |
— | UNKNOWN_WORKER |
simlock worker update <disconnected id> |
— | WORKER_NOT_CONNECTED: worker <id> is not connected |
simlock worker update <id> --detach |
— | Updating mac-studio-2 (3f81a2c4) 0.3.0 → 0.4.0. Follow it with simlock worker list. |
simlock worker update <id>, already latest |
— | mac-studio-2 (3f81a2c4) is already on the latest (0.4.0) |
simlock worker update on a worker daemon |
— | UNSUPPORTED_IN_WORKER_MODE |
simlock update --gateway |
— | installs and restarts at once; a request in that gap fails as in any gateway restart |
If this task goes wrong, a user would notice a worker kept out of rotation after its update, leases sent to an updating worker, or an operator's drain undone.
Technical spec
Facts
| # | Fact | Source |
|---|---|---|
| F1 | The gateway keeps the worker's client even when status.get answers PROTOCOL_VERSION_UNSUPPORTED, and marks the link incompatible |
src/gateway/worker-link.ts:165-167, 198, 210-221 |
| F2 | The degraded client lets daemon.update through (#836) |
#836 Tests |
| F3 | The 30 s tick refreshes every link, incompatible ones included; WORKER_CALL_TIMEOUT_MS (10 s) bounds view reads only |
src/gateway/service.ts:287; src/gateway/worker-link.ts:44, 184 |
| F4 | A refresh merges into the old view; an absent key keeps its value (the new fields are never absent) | src/gateway/worker-registry.ts:345-365 |
| F5 | The routing filter skips a worker not connected, then a drained one; reasons in COULD_TAKE_LATER keep a request waiting |
src/gateway/routing/stages/takes-requests.ts:14-15; src/gateway/fleet-coordinator.ts:117-122 |
| F6 | A worker answer other than NO_CAPACITY, PROVISION_FAILED or a cannot-serve code ends a fleet request worker-failed |
src/gateway/fleet-coordinator.ts:1053-1096 |
| F7 | worker.drain/remove handlers are the models; their tests are in src/gateway/dispatcher.test.ts:322-331 and src/daemon/gateway-fleet.e2e.test.ts; a worker answers worker.* writes with UNSUPPORTED_IN_WORKER_MODE |
src/gateway/dispatcher.ts:228-231, 375-397; src/daemon/dispatcher.ts:327-329 |
| F8 | worker.* inputs use workerId: z.string().min(1); UNKNOWN_WORKER exists, WORKER_NOT_CONNECTED doesn't |
src/contract/operations.ts:753; src/contract/errors.ts:120-124 |
| F9 | Waiting-reason texts are Records over the reason enum: GATEWAY_REASON_TEXT in the CLI and WORKER_WHY_TEXT in the console |
src/cli/commands/list.ts:62-76; ui/src/model/waiting.ts:96-108 |
| F10 | The worker list line is id (label): state -- capacity, leases[…] |
src/cli/format.ts:5-41 |
| F11 | The worker routes are at app.ts:661-691, declared by #821 |
src/http/app.ts:661-691; #821 |
| F12 | The gateway's own daemon object is built in aggregate.ts and the gateway dispatcher |
src/gateway/aggregate.ts:67; src/gateway/dispatcher.ts:317 |
| F13 | isNewerVersion is #837's |
#837 |
Modules touched
Searches: rg "drained" in src/gateway, rg "GatewayWaitingReason" (contract, cli list, ui waiting, pipeline), rg "formatWorkers", rg "\"worker\\.drain\"".
Files: src/contract/{operations,schemas,errors,worker-view,protocol}.ts, operations.test.ts; src/gateway/{dispatcher,worker-link,worker-registry,service,aggregate,test-support}.ts, src/gateway/routing/{pipeline.ts,stages/takes-requests.ts}, src/gateway/AGENTS.md; src/daemon/{main,dispatcher}.ts (the gateway role of daemon.update, wired through main.ts); src/simlock-client/{client,types}.ts, src/admin/index.ts; src/cli/commands/{worker,list}.ts, src/cli/format.ts; ui/src/model/waiting.ts (the reason text only); src/http/app.ts; docs CLI.md, HTTP-API.md, CLIENT.md.
Contract and event changes
This PR bumps the protocol by one.
worker.update: admin, write. Input{ workerId: z.string().min(1), version?, force? }with #836's bounds. The answer is #836's answer plusworkerId. On the uplink it's bounded at 60 s. At the limit, the caller getsWORKER_NOT_CONNECTEDwithworker <id> did not answer within 60 s; check simlock worker list, and the view catches up on the next read.- For an
incompatibleworker,daemon.update {query}runs on the 30 s tick and right afterworker.update. A failed query keeps the last view. - Worker view:
update,lastUpdateFailure,latestVersion, eachnullwhen there's nothing to show. - Waiting reason
updating. Its text isUpdatingin both Records. It's inCOULD_TAKE_LATER, since the worker comes back by itself. WORKER_NOT_CONNECTED(domain, 409, exit 2):worker <id> is not connected.- A status re-read also runs on
daemon.update-startedanddaemon.update-failed(besideWORKER_VIEW_CATALOG_EVENTS) and right afterworker.updateanswers. daemon.updateon a gateway: no lease wait, straight toinstalling; fleet admission is not changed.
Other code on the same state
Searches: rg "setDrained" (registry, dispatcher), rg "takesRequests" (routing 3).
- The drain set and
updateare separate. An update never callssetDrained, and a drain or undrain during an update changes only the drain set. - A lease-event refresh re-reads
status.get, and itsupdate: nullclears the view.
Failure modes
| Call | Outcome | What happens | Limit, counted from |
|---|---|---|---|
daemon.update over the uplink |
ok | view updated from the answer | 60 s |
| same | error reply | passed to the caller (UPDATE_*) |
— |
| same | uplink closed / timeout | WORKER_NOT_CONNECTED / the timeout error |
60 s |
simlock worker update follow |
view version = to, update null, connected or incompatible |
… is running <to>, exit 0 |
none; Ctrl-C stops following |
| same | lastUpdateFailure.to = to |
Update failed (<stage>): <message>, exit 1 |
— |
| gateway's own install | fails | lastUpdateFailure in its status.get, it keeps serving |
npm's 10 min |
| gateway's new version | doesn't start | #835 rolls back; the gateway's start reads <root>/gateway/update-result.json (#836's readResultFile, wired in startGatewayDaemon) and reports it |
2 min (#835) |
A gateway restart mid-update rebuilds each view from the worker, which still reports update.
Rules in play
- ADR 0042 §2: "
worker.update… sendsdaemon.updateover that worker's uplink". - ADR 0042 §3 and ADR 0005 §9: the update never touches the drain set, and "only
undrainends it". - ADR 0005 §31: an incompatible worker "is never dispatched to". Only the frozen call goes out.
Tests
Seams: src/gateway/dispatcher.test.ts and src/daemon/gateway-fleet.e2e.test.ts with ScriptedWorkerClient (src/gateway/test-support.ts:271, given a pass-through for daemon.update when degraded), modelled on the drain tests. Routes go through src/http/app.test.ts, the gateway's own update through src/daemon/main.test.ts, and the CLI through src/cli/index.test.ts.
- Operator calls
worker.updateon the gateway: connected (worker getsdaemon.update, viewupdateset); incompatible (frozen call sent, view filled fromquery); disconnected (WORKER_NOT_CONNECTED); unknown id (UNKNOWN_WORKER);POST /v1/workers/{id}/update(202, 200, 403 for an agent token). - Gateway routes a lease while a worker is updating: an updating worker is skipped (
updating, the request waits); the update ends (update: nullclears the view, the worker is picked again); an operator-drained worker stays drained after its update. - Operator updates the gateway itself: no wait with fleet leases held; a failed install shows in its
status.get; a rolled-back gateway reports the failure at start. - Operator runs
simlock worker updateandsimlock worker list: follows to… is running <v>; failure (exit 1); already latest;--detach; on a worker daemon (UNSUPPORTED_IN_WORKER_MODE); the list line showssimlock <v> (<latest> available)only whenisNewerVersion, andupdating to <v>: <stage>.
Done when
pnpm test:changedpasses (it runs the unit tests under src/gateway src/daemon src/cli src/http src/contract,src/daemon/gateway-fleet.e2e.test.tsincluded).- CI's check job passes.
docs/CLI.mddocumentssimlock worker updateand the new list fields;docs/HTTP-API.mddocuments the routes.
Out of scope
- Updating every worker in one call. Queuing an update for a disconnected worker. The console (#839).
Depends on
- #837
- #821
Written by an agent.
- Dominant language
- TypeScript
- Stars
- 20
- Forks
- 2
- Avg merge
- 8h
- Merged PRs (30d)
- 316
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from callstackincubator/simlock
-
bug:new
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
callstackincubator/simlock#817 ·
Maintainers usually reply within 1 day
-
bug:triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
callstackincubator/simlock#778 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
callstackincubator/simlock#639 · 1 comment ·
Maintainers usually reply within 1 day
-
task:draft
Difficulty 4/5 3-5 days Newbie friendliness 35/100
callstackincubator/simlock#839 ·
Maintainers usually reply within 1 day
-
task:ready
Difficulty 4/5 3-5 days Newbie friendliness 57/100
callstackincubator/simlock#837 ·
Maintainers usually reply within 1 day
All issues in callstackincubator/simlock
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 72/100
zts212653/clowder-ai#1678 ·
Maintainers usually reply within 1 day
-
good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
Progressive-Victory/the-website#680 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100