Bogdanp/django_dramatiq

How to use two queues with different configurations for processes and threads ?

Ouverte

#75 ouverte le 13 août 2020

 (0 commentaire) (5 réactions) (0 personne assignée)Python (86 forks)github user discovery
help wanted

Métriques du dépôt

Stars
 (384 étoiles)
Métriques de merge PR
 (Aucune PR mergée en 30 j)

Description

I need to run one queue exclusively with one thread and one process to enforce linearization. I run a docker container with:

./manage.py rundramatiq --processes 1 --threads 1 -Q default

However I need a second queue for notifications that is not restricted so I run a second container with:

./manage.py rundramatiq --processes 1 --threads 4 -Q notifications

The tasks work but this kind of setup is creating a collision on the task id at the Database level.

postgres_1            | 2020-08-13 21:19:44.423 UTC [686] ERROR:  duplicate key value violates unique constraint "django_dramatiq_task_pkey"
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] DETAIL:  Key (id)=(2ac20d90-dac0-40ec-9192-e52468c21faf) already exists.
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] STATEMENT:  INSERT INTO "django_dramatiq_task" ("id", "status", "created_at", "updated_at", "message_data", "actor_name", "queue_name") VALUES ('2ac20d90-dac0-40ec-9192-e52468c21faf'::uuid, 'enqueued', '2020-08-13T21:19:44.371334+00:00'::timestamptz, '2020-08-13T21:19:44.373001+00:00'::timestamptz, '..... '::bytea, message_user', 'notifications')

Any ideas on how to setup this for my requirement and why is creating a collision when generating the ID ?

Guide contributeur