How to solve the race condition problem in Pool multiprocessing
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- postgresql, python
- Domain
- databases, distributed-systems
Research direction
The issue names billiard Pool, multiprocessing.Process/Lock, and psycopg2, but no repository file or failing test. Start by reproducing the reported connection get/close failure and compare billiard's synchronization support with multiprocessing.Lock. Done is not defined beyond finding whether a safe locking approach exists.
Written by the indexing model from the issue text.
Description
Hi,
When I am getting and closing DB connection (psycopg2) in the processes of billiard Pool, I found that getting and closing of DB connection will fail. Is there any way that I can add some lock to the getting and closing methods such that the race condition problem can be avoid?
In multiprocessing, a Lock object is provided. I wonder if there are something similar in billiard.
from multiprocessing import Process, Lock
def f(l, i):
l.acquire()
try:
print('hello world', i)
finally:
l.release()
if __name__ == '__main__':
lock = Lock()
for num in range(10):
Process(target=f, args=(lock, num)).start()
- Dominant language
- Python
- Stars
- 436
- Forks
- 273
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 5
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from celery/billiard
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
return in finally can swallow exceptionsMay be free again @auvipy claimed this 602 days ago, and no pull request is open. Open
celery/billiard#414 · 6 comments · 3 reactions · 1 assignee ·
-
Handling numpyOpen
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
AI bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
AUTO-MAS-Project/AUTO-MAS#1960 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
btclib-org/btclib-wallet#872 ·
Maintainers usually reply within 1 day
-
[Bug]: Scheduled returns scan all accepted minion keys causing extreme loadPossibly taken A pull request linked to this issue is open or already merged. Openbug needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
saltstack/salt#70789 · 1 comment ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
btclib-org/btclib#3093 ·
Maintainers usually reply within 1 day