Additional SQS retry strategy for failed batches
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- aws, javascript
- Domain
- cloud
Research direction
Start by locating the failedBatchReprocessingLambda and the existing SNS failure-event handling. Trace how failed batches are received and retried, then determine how both SNS and SQS event shapes should be supported while preserving immediate failure notification and delayed reprocessing. Done means the Lambda handles the requested SQS subscription flow and its behavior is covered by the repository's existing tests.
Written by the indexing model from the issue text.
Description
The failedBatchReprocessingLambda provides for the option of subscribing to an SNS topic to reprocess failed batches.
I would like to defer attempts to re-process failed batches, for example during Redshift Maintenance Windows where immediately re-attempting to re-process batches will not make sense. There are SNS delivery policies such as minimum delay that could achieve this aim. The challenge is that I would like a subscriber that receives the failure event without delay but re-process the batch with a delay.
This could be achieved by subscribing an SQS queue to the subscriber and enhancing the failedBatchReprocessingLambda to be able to process SQS OR SNS events. In this way I could have one subscriber that is notified without delay and then set a "Delivery Delay" on an SQS queue. I would like to subscribe the failedBatchReprocessingLambda to the SQS queue that contains the original SNS failure message.
- Dominant language
- JavaScript
- Stars
- 595
- Forks
- 161
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from awslabs/aws-lambda-redshift-loader
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 32/100
awslabs/aws-lambda-redshift-loader#247 · 3 comments ·
-
Manifest missing file returned in describeBatch.jsMay be free again @IanMeyers claimed this 1379 days ago, and no pull request is open. Open
awslabs/aws-lambda-redshift-loader#244 · 4 comments · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
awslabs/aws-lambda-redshift-loader#239 · 4 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
awslabs/aws-lambda-redshift-loader#238 · 1 comment ·
All issues in awslabs/aws-lambda-redshift-loader
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
daisy/a11y-meta-viewer#18 ·
-
good first issue status: needs triaging type: bug version: 2.0
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
medusajs/medusa#17094 · 2 comments ·
Maintainers usually reply within 1 day
-
browser: chrome package: @carbon/react package: styles
Difficulty 1/5 Under an hour Newbie friendliness 92/100
carbon-design-system/carbon#23567 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
clerk/javascript#10033 ·
Maintainers usually reply within 1 day
-
bug client p1
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vercel/eve#4173 · 2 comments ·
Maintainers usually reply within 1 day