limits_concurrency on_conflict: :discard looking only into running jobs and not blocked jobs
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 52/100
Research direction
Start at the concurrency handling for limits_concurrency with on_conflict: :discard, and trace how jobs in Running, Ready, and Blocked are considered while a semaphore remains active. Reproduce the ProcessPrunedError-style state with frequent scheduling; done means conflicting jobs are discarded rather than accumulating in Blocked until the semaphore expires.
Written by the indexing model from the issue text.
Description
In the uncommon case that (because of whatever error, possibly ProcessPrunedError), a job is not running anymore but the Semaphore hasn't expired yet, SolidQueue is piling up new jobs in "Blocked", thus not fulfilling the on_conflict: discard promise.
I see two possible solutions:
- Consider checking if a semaphore is blocking for the key
- Check if a job is "running" not only by looking if there is a job in running or ready but also possibly in blocked
It's slightly an edge case, but if the cron is set up to a quite high frequency, it can pile up quite a bit in the few hours the semaphore hasn't expired.
NB: this happened on SolidQueue 1.3.1. We want to update soon and I have not seen in the changelog what could have fixed it "en passant"
- Dominant language
- Ruby
- Stars
- 2.5k
- Forks
- 250
- Avg merge
- 8h 31m
- Merged PRs (30d)
- 5
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from rails/solid_queue
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
rails/solid_queue#802 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
rails/solid_queue#797 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
rails/solid_queue#792 · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
rails/solid_queue#780 ·
-
Removing job from recurring.yml does not remove job from recurring tasks tab in mission control Open
Difficulty 3/5 1-2 days Newbie friendliness 45/100
rails/solid_queue#623 · 2 reactions ·
All issues in rails/solid_queue
Similar issues
-
user-reported
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Kong/developer.konghq.com#7316 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
TheOdinProject/curriculum#31408 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bensheldon/good_job#1816 · 5 comments ·