EMR executor market type is not validated and any value other than ON_DEMAND is treated as SPOT
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
Research direction
Find the table property definition for sleeper.table.bulk.import.emr.executor.market.type and compare its validation with sleeper.default.table.bulk.import.emr.executor.market.type. Then inspect BulkImportExecutor.validateJob for platform spec overrides; add validation in both paths and confirm invalid values are rejected while ON_DEMAND and SPOT are accepted.
Written by the indexing model from the issue text.
Description
Description
The table property sleeper.table.bulk.import.emr.executor.market.type says the valid values are ON_DEMAND or SPOT, but it has no validation predicate. The instance property sleeper.default.table.bulk.import.emr.executor.market.type does have one. The value can also be set in the platform spec of a bulk import job, which is not validated either.
In EmrInstanceFleets we check "ON_DEMAND".equals(...) and use spot capacity otherwise, so a value like on_demand gives spot executors without any error. In EmrInstanceGroups, MarketType.fromValue turns an unknown value into UNKNOWN_TO_SDK_VERSION.
Technical Notes / Implementation Details
Add the same validation as the instance property to the table property, so that only ON_DEMAND or SPOT are accepted.
The market type can also be overridden per job in the platform spec, which doesn't go through property validation. We should also check it in BulkImportExecutor.validateJob, so a job with an invalid market type is rejected
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 1d 33m
- Merged PRs (30d)
- 115
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from gchq/sleeper
-
enhancement scripts
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
bug needs-refinement under-discussion
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
gchq/sleeper#8632 · 1 comment ·
Maintainers usually reply within 1 day
-
bug needs-refinement under-discussion
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Transaction log follower logs an error for every transaction deleted from the files logPossibly taken @ca61688 claimed this today. Openbug size:small
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
gchq/sleeper#8624 · 2 comments · 1 assignee ·
Maintainers usually reply within 1 day
-
bug size:medium
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
SasanLabs/VulnerableApp#912 ·
Maintainers usually reply within 1 day
-
area/build area/events area/storage area/storage/kafkasql area/storage/sql
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Apicurio/apicurio-registry#11079 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
OpenAEV-Platform/openaev#9139 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
desktop edt not responding
Difficulty 2/5 1-3 hours Newbie friendliness 78/100