SIGTERM kills db-sync mid-pull; proposal: finish the current connection and exit between cycles
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 52/100
Research direction
Start by reading dbsync_daemon.py and tracing dbsync_pull(), dbsync_push(), and run_daemon() to understand the cycle boundaries described in the issue. Implement the proposed SIGTERM shutdown behavior, then verify that a started pull or push can finish before the process exits and that SIGINT retains its default behavior.
Written by the indexing model from the issue text.
Description
dbsync_daemon.py installs no SIGTERM handler, so a docker stop,
docker compose up -d after a rebuild, or a service manager restart ends
the process wherever it happens to be. If that is inside pull() between
"GeoPackage updated" (the checkout is at the new version) and "base
schema updated" (the changeset has been applied to the database), the
checkout and the base schema are at the same version with different
content, and the next start refuses with
The db schemas already exist but 'base' schema is not synchronized with source GPKG. Running `dbsync_deamon.py` with `--force-init` should fix the issue.
--force-init then drops and rebuilds the schemas, which loses any
not-yet-pushed database edits and any views or grants on the schemas.
We hit this in production with four projects and a routine image
rebuild (one daemon was pulling 48 files at that moment).
Proposal, implemented in the PR that follows: a SIGTERM handler that only
sets a flag; dbsync_pull()/dbsync_push() check it before each
connection; run_daemon() checks it at the top of the cycle and between
pull and push; the sleep polls it once a second. A pull or push that has
started always completes; the process exits 0 at the next boundary.
SIGINT keeps its default so ctrl+c still kills immediately. A container
needs stop_grace_period long enough for one connection's pull
(we use 150 s).
- Dominant language
- Python
- Stars
- 55
- Forks
- 25
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 1
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MerginMaps/db-sync
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
MerginMaps/db-sync#184 ·
-
Rebase: a field update of a row deleted in the database is dropped without any record; and the conflicts file is per project, not per connectionPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
MerginMaps/db-sync#195 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 38/100
MerginMaps/db-sync#185 · 1 comment ·
-
db-sync enters an infinite drop/recreate loop when geodiff init fails on invalid source geometriesOpen
Difficulty 4/5 3-5 days Newbie friendliness 42/100
MerginMaps/db-sync#182 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 52/100
MerginMaps/db-sync#181 · 2 comments ·
All issues in MerginMaps/db-sync
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Deltares/numba_celltree#100 · 1 comment ·
-
auto-reported bug
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
Maintainers usually reply within 1 day
-
soup airgap-bundle --bundle-size-cap nan, inf or a cap above ~1.67e299 GiB stops with an internal ValueError/OverflowError (exit 1) instead of a usage errorPossibly taken @Keshemir claimed this today. Openbug claimed difficulty:2
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
MakazhanAlpamys/Soup#3045 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
dagster-io/skills#80 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day