files not loaded occasionally due "another thread has locked it for loading"
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- aws, javascript
Research direction
No source file or test is named. Start by tracing the batch-loading path that emits “another thread has locked it for loading” and handles ConditionalCheckFailedException, using the reported timeout, batch size, and batchSizeBytes settings. Done means the intermittent skipped-file cause is identified and a verified mitigation or fix is documented.
Written by the indexing model from the issue text.
Description
A small number of my files (around 10 out of 20k) is not loaded each day, with such error being returned after it hits the timeout:
error: The conditional request failed
{
"code": "ConditionalCheckFailedException",
"time": "2021-08-23T21:16:14.506Z",
"requestId": "5UCMPG1CQ6QRQAATPJPH8V536VVV4KQNSO5AEMVJF66Q9ASUAAJG",
"statusCode": 400,
"retryable": false,
"retryDelay": 25.6182288251398
}
and logging this:
debug: Batch is ready to be processed, but another thread has locked it for loading
Despite the debug log above, I wasn't able to find any other batches with these files. My configuration is:
batchTimeoutSecs: 240batchSize: 100batchSizeBytes: ""
I wonder if anyone else experienced similar issue and whether there's a simple config tweak that could be applied to mitigate this? Thanks.
- Dominant language
- JavaScript
- Stars
- 595
- Forks
- 161
- PR merge metrics
- No merged PRs in 30d
Getting set up
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from awslabs/aws-lambda-redshift-loader
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 32/100
awslabs/aws-lambda-redshift-loader#247 · 3 comments ·
-
Manifest missing file returned in describeBatch.jsMay be free again @IanMeyers claimed this 1376 days ago, and no pull request is open. Open
awslabs/aws-lambda-redshift-loader#244 · 4 comments · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
awslabs/aws-lambda-redshift-loader#239 · 4 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
awslabs/aws-lambda-redshift-loader#238 · 1 comment ·
All issues in awslabs/aws-lambda-redshift-loader
Similar issues
-
Add: Digi 4K LogoOpencheck:passed logos:add
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
component/administration domain/framework needs-triage Platform(Default) priority/low
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
semantic-release/gitlab#1032 ·
-
agent/guide documentation hive/hosted-available-lke648397-260827-5n31
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Maintainers usually reply within 1 day
-
agentic-workflows
Difficulty 1/5 Under an hour Newbie friendliness 85/100
githubnext/gh-aw-workshop#3933 ·
Maintainers usually reply within 2 days