Poison messages handling
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- csharp
- Domain
- backend, distributed-systems
Research direction
Start by tracing the PoisonMessageDetected and AbandoningMessage log paths, using issue #1039 for context. Verify the reported 600ms versus 600s wording and determine how poison-message dequeue limits and disposal should be specified; done means the behavior and visibility-delay message match the agreed handling.
Written by the indexing model from the issue text.
Description
As a rokochet of issue #1039 I've noticed that the message is being handled indefinitely.
I see the following traces in my logs:
PoisonMessageDetected: d8bdcf4d-ef40-4501-a99b-6e72309644ee: Message [TaskScheduled#1] with ID 6128efd4-8904-494d-9a2d-d2d8d01edbe7 has been dequeued 79 times and is now considered poison: {"Account":"...","TaskHub":"Services","...":"TaskScheduled","TaskEventId":1,"MessageId":"6128efd4-8904-494d-9a2d-d2d8d01edbe7","InstanceId":"d8bdcf4d-ef40-4501-a99b-6e72309644ee","ExecutionId":"b27db20915cf4e5db6cad3589dde88df","PartitionId":"...-workitems","DequeueCount":79}
AbandoningMessage: d8bdcf4d-ef40-4501-a99b-6e72309644ee: Abandoning [TaskScheduled#1] message back to ...-workitems and setting a visibility delay of 600ms: {"Account":"...","TaskHub":"...","EventType":"TaskScheduled","TaskEventId":1,"MessageId":"6128efd4-8904-494d-9a2d-d2d8d01edbe7","InstanceId":"d8bdcf4d-ef40-4501-a99b-6e72309644ee","ExecutionId":"b27db20915cf4e5db6cad3589dde88df","PartitionId":"services-workitems","SequenceNumber":208,"PopReceipt":"AgAAAAMAAAAAAAAAxVkNR+xc2gE=","VisibilityTimeoutSeconds":600}
Side note: the message say
setting a visibility delay of 600ms, while the code actually delays for 600s (not ms).
Notice the DequeueCount value of 79 (I've seen values much higher). Perhaps there should be a setting that controls the maximum number of dequeues attempts, after which the message should just be disposed?
@cgillum @davidmrdavid
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 335
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 8
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Azure/durabletask
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Azure/durabletask#1398 · 2 comments ·
Maintainers usually reply within 3 days
-
Azure Storage backend: control queue partition left unowned for hours/days after lease expiresMay be free again @nytian claimed this 31 days ago, and no pull request is open. Open
Azure/durabletask#1389 · 1 comment · 1 assignee ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Azure/durabletask#1332 ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 1-2 days Newbie friendliness 58/100
Azure/durabletask#1318 · 1 comment ·
Maintainers usually reply within 3 days
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Azure/durabletask#1301 · 3 comments ·
Maintainers usually reply within 3 days
All issues in Azure/durabletask
Similar issues
-
copilot documentation
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
[Rust][Flaky Test] multiple_deadlines_fire_in_order asserts a wall-clock gap instead of firing orderOpenCI/CD ⚒️ Flaky-tests 🐦
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
valkey-io/valkey-glide#7255 ·
Maintainers usually reply within 3 days
-
bug good first issue
Difficulty 1/5 Under an hour Newbie friendliness 92/100
unoplatform/Uno.Core#99 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
aws/aws-dotnet-ai#75 ·