Bogdanp/django_dramatiq

How to use two queues with different configurations for processes and threads ?

Open

#75 opened on Aug 13, 2020

 (0 comments) (5 reactions) (0 assignees)Python (86 forks)github user discovery
help wanted

Repository metrics

Stars
 (384 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

I need to run one queue exclusively with one thread and one process to enforce linearization. I run a docker container with:

./manage.py rundramatiq --processes 1 --threads 1 -Q default

However I need a second queue for notifications that is not restricted so I run a second container with:

./manage.py rundramatiq --processes 1 --threads 4 -Q notifications

The tasks work but this kind of setup is creating a collision on the task id at the Database level.

postgres_1            | 2020-08-13 21:19:44.423 UTC [686] ERROR:  duplicate key value violates unique constraint "django_dramatiq_task_pkey"
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] DETAIL:  Key (id)=(2ac20d90-dac0-40ec-9192-e52468c21faf) already exists.
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] STATEMENT:  INSERT INTO "django_dramatiq_task" ("id", "status", "created_at", "updated_at", "message_data", "actor_name", "queue_name") VALUES ('2ac20d90-dac0-40ec-9192-e52468c21faf'::uuid, 'enqueued', '2020-08-13T21:19:44.371334+00:00'::timestamptz, '2020-08-13T21:19:44.373001+00:00'::timestamptz, '..... '::bytea, message_user', 'notifications')

Any ideas on how to setup this for my requirement and why is creating a collision when generating the ID ?

Contributor guide