world-postgres: let close() give running deliveries more than graphile's fixed 5 s before aborting them

Open
#5,152 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
postgresql, typescript

Research direction

Start by reading PostgresWorldConfig and createQueue, which builds Graphile's run() options; then check how close() calls runner.stop() and how the existing shutdown tests cover running deliveries. Done when the configured grace period reaches Graphile's shutdown timeout, defaults remain compatible, and tests verify deliveries finish within the grace period or are aborted afterward.

Written by the indexing model from the issue text.

Description

Summary

An application-managed shutdown (applicationManagedShutdown: true, then await world.close()) still cuts off any step that has been running for more than 5 seconds when the process is told to stop.

close() calls runner.stop(), which runs graphile-worker's workerPool.gracefulShutdown(). That aborts every running job's abortSignal after gracefulShutdownAbortTimeout, which graphile-worker 0.16.6 fixes at 5 s in its defaults (dist/config.js). world-postgres passes that signal to the loopback delivery (dist/queue.js: "Queue shutdown aborts the delivery through Graphile's signal"), so the delivery is cancelled, the job is failed for a retry, and close() goes on to close the streamer and the pool. Once the process exits, the step that was running is replayed only after the inline ownership lease (WORKFLOW_INLINE_OWNERSHIP_LEASE_SECONDS, 860 s by default) runs out.

On Kubernetes a terminating pod gets a grace period, often 30 to 120 s. Most agent steps (one model call and its tools) finish inside that, but few finish inside 5 s, so every rolling deploy stalls each in-flight step for a lease length.

Environment

  • @workflow/world-postgres 5.0.5, graphile-worker 0.16.6 (transitive)
  • An agent served with eve start (eve 0.90), Node 24, Kubernetes with a 120 s termination grace period

Ask

A way to set how long close() lets running deliveries finish before it aborts them, for example a shutdownGraceMs option on PostgresWorldConfig (and a WORKFLOW_POSTGRES_SHUTDOWN_GRACE_MS default), passed to graphile-worker as preset.worker.gracefulShutdownAbortTimeout. Today createQueue builds graphile's run() options itself (concurrency, logger, noHandleSignals, pollInterval, taskList), so an application can't pass a preset.

With that, and a server that waits for close() before exiting (vercel/eve#1981, draft vercel/eve#1983), a rolling deploy could stop claiming jobs, let in-flight steps finish inside the platform's grace period, and release only what is still running at the end.

Related

  • #4244: close() resolving while a job is still running, fixed on the 5.x line.
Dominant language
TypeScript
Stars
2.5k
Forks
401
Avg merge
1d 21h
Merged PRs (30d)
201

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vercel/workflow

All issues in vercel/workflow

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.