Reference specs: document signal handling in populate() (SIGTERM mechanics)

Open Beginner friendly
#262 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
documentation

Research direction

Read src/reference/specs/autopopulate.md and job-metadata.md, then verify the described behavior against autopopulate.py and the worker code. Add a “Signals and interruption” section covering handler scope, transaction and job-state consequences, redo versus lost work, and orphaned-job reclaim via refresh(orphan_timeout=...).

Written by the indexing model from the issue text.

Description

The job-reservation reference specs (src/reference/specs/autopopulate.md, job-metadata.md) say nothing about signal handling, yet datajoint-python has specific, load-bearing SIGTERM behavior during populate() that operators of distributed workers depend on:

  • populate() installs a SIGTERM handler for the duration of each cycle; a SIGTERM mid-make() raises SystemExit, which escapes except Exception (so suppress_errors=True does not swallow it), cancels the open transaction, and records the job as error with a %SIGTERM% message.
  • %SIGTERM% is in the permanent-error patterns, so the worker's next cycle deletes the matching error row and the key re-queues — the interrupted work is redone, not lost.
  • By contrast, a signal with no installed handler (default disposition: terminate) kills the process outright: no transaction cancel, no jobs.error() row — the job stays reserved until reclaimed via refresh(orphan_timeout=...).

None of this is derivable from the current specs, and it matters for anyone designing worker lifecycle management, container orchestration (SIGTERM on pod eviction), or graceful-drain schemes on top of DataJoint. Suggested home: a "Signals and interruption" section in autopopulate.md covering the handler's scope, the transaction/job-state consequences, the redo-vs-lost distinction, and the orphaned-job reclaim path. Verify the exact mechanics against autopopulate.py/the worker code at writing time rather than from this summary.

Dominant language
Jupyter Notebook
Stars
7
Forks
12
Avg merge
3d 8h
Merged PRs (30d)
12

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from datajoint/datajoint-docs

All issues in datajoint/datajoint-docs

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.