Bogdanp/django_dramatiq

How to use two queues with different configurations for processes and threads ?

Aberta

#75 aberto em 13 de ago. de 2020

 (0 comentário) (5 reações) (0 responsável)Python (86 forks)github user discovery
help wanted

Métricas do repositório

Stars
 (384 estrelas)
Métricas de merge de PR
 (Nenhuma PRs mesclada em 30d)

Description

I need to run one queue exclusively with one thread and one process to enforce linearization. I run a docker container with:

./manage.py rundramatiq --processes 1 --threads 1 -Q default

However I need a second queue for notifications that is not restricted so I run a second container with:

./manage.py rundramatiq --processes 1 --threads 4 -Q notifications

The tasks work but this kind of setup is creating a collision on the task id at the Database level.

postgres_1            | 2020-08-13 21:19:44.423 UTC [686] ERROR:  duplicate key value violates unique constraint "django_dramatiq_task_pkey"
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] DETAIL:  Key (id)=(2ac20d90-dac0-40ec-9192-e52468c21faf) already exists.
postgres_1            | 2020-08-13 21:19:44.423 UTC [686] STATEMENT:  INSERT INTO "django_dramatiq_task" ("id", "status", "created_at", "updated_at", "message_data", "actor_name", "queue_name") VALUES ('2ac20d90-dac0-40ec-9192-e52468c21faf'::uuid, 'enqueued', '2020-08-13T21:19:44.371334+00:00'::timestamptz, '2020-08-13T21:19:44.373001+00:00'::timestamptz, '..... '::bytea, message_user', 'notifications')

Any ideas on how to setup this for my requirement and why is creating a collision when generating the ID ?

Guia do colaborador