Review exception handling in ingest/bulk import
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 45/100
Research direction
Start by reading BulkImportStarterLambda, IngestTask, IngestJobQueueConsumer, and their shared IngestJobMessageHandler to trace how exceptions are handled and ingest jobs are tracked. Compare the required failure behavior in the starter lambda and ingest task, then identify a way to make the handling easier to verify. Done when failures are tracked and caught or rethrown appropriately in both contexts, with verification covering those behaviors.
Written by the indexing model from the issue text.
Description
Description / Background
As part of the following PR we noticed some difficulties in determining the correctness of exception handling in the bulk import starter lambda and in ingest tasks:
We want to be confident that all failures in the bulk import starter lambda, and in ingest tasks, will both be tracked in the ingest job tracker, and will be either caught or rethrown as appropriate for its context. We would like to make this easier to verify if possible.
We would like to spend at most a day or so on this for this issue.
Technical Notes / Implementation Details
This is in BulkImportStarterLambda and IngestTask/IngestJobQueueConsumer.
Part of this is that they both share IngestJobMessageHandler, but they have different needs for handling exceptions. In the bulk import starter lambda, an uncaught exception will result in a single failed job, which can be retried by SQS. In the ingest task, a completely uncaught exception will stop the whole task.
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 1d 33m
- Merged PRs (30d)
- 115
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from gchq/sleeper
-
enhancement scripts
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
EMR executor market type is not validated and any value other than ON_DEMAND is treated as SPOTOpenbug bulk-import-module
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bug needs-refinement under-discussion
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
gchq/sleeper#8632 · 1 comment ·
Maintainers usually reply within 1 day
-
bug needs-refinement under-discussion
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Transaction log follower logs an error for every transaction deleted from the files logPossibly taken @ca61688 claimed this 1 day ago. Openbug size:small
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
gchq/sleeper#8624 · 2 comments · 1 assignee ·
Maintainers usually reply within 1 day
Similar issues
-
type: possible bug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
-
面渣逆袭Spring事务传播机制勘误Open
Difficulty 1/5 Under an hour Newbie friendliness 80/100
itwanger/toBeBetterJavaer#218 ·
-
Markdown and Astro language servers do not start when the project path contains a non-ASCII character or a spacePossibly taken @RoiSoleil claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
eclipse-wildwebdeveloper/wildwebdeveloper#2277 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
johanhaleby/occurrent#1462 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
SasanLabs/VulnerableApp#912 ·
Maintainers usually reply within 1 day