Identify database race conditions
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start by reviewing the database operations, especially update operations, in the internal dynamo library at lib/dynamo/dynamo; trace multi-query flows and credit-related operations for possible race conditions. Then inspect production API Lambda logs for DatabaseConditionException occurrences since early 2024. The issue is complete when potential race conditions or failing condition checks are documented as follow-up issues, or the review finds none.
Written by the indexing model from the issue text.
Description
Jira: https://asfdaac.atlassian.net/browse/TOOL-4691
Note: The above link is accessible only to members of ASF.
It may be worthwhile to review our internal dynamo library for potential DynamoDB race conditions, especially if we ever offer a pay-for-processing deployment in which users are actually paying for credits.
We already attempt to guard against the obvious cases via use of ConditionExpression in many of our queries, but there could be subtler race conditions, especially when multiple database queries are involved; for example, see https://github.com/ASFHyP3/hyp3/issues/2720 (not really a race condition itself, but it's a consequence of a change we made to address a race condition, details in the issue).
It's entirely possible that we won't identify any important issues. But it still might be worth reviewing our database operations (particularly update operations) and creating follow-up issues for potential race conditions, if any are found, especially where the user's credits are concerned.
It might also be worth doing a query of prod HyP3's API Lambda logs for occurrences of DatabaseConditionException (which is our custom exception for failed ConditionExpression checks and appears as an Internal Server Error for the user) since early 2024 (which I believe is when we added that exception type), to see if any of our existing ConditionExpression checks have been failing.
- Dominant language
- Python
- Stars
- 53
- Forks
- 13
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 20
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ASFHyP3/hyp3
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Most static tests fail when a date-like string is provided as an example value in a job spec YAMLOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Jira Bug
Difficulty 1/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
aws-samples/sample-per-model-bedrock#857 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
521xueweihan/HelloGitHub#4328 ·
-
documentation maintainer-managed
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
awslabs/visual-asset-management-system#1418 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day