Investigate AWS lambda execution
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
Research direction
Start by reviewing issue #3's windowed-read work and the Counts processing entry point; the payload names no files or tests. Investigate whether GDAL and its requirements fit within AWS Lambda across the listed memory configurations, and treat a deployed, measured comparison of those configurations and request routing as done.
Written by the indexing model from the issue text.
Description
Brainstorming.
If #3 works well for windowed reads, create a lambda function for Counts and deploy to a variety of memory configurations (ie, 0.5GB, 1GB, 1.5GB). Route requests through a load balancer which can take the input geometry and the target raster metadata (cell size + data type) and evaluate the amount of memory needed to read in the bounding box window - then route to that configured function for processing.
Pros:
- Scalable for concurrent requests, no server to overwhelm
- Minimizes cost per request
- Could scale to many generic functions, ie
Counts,MapAlgebra_Add, etc
Cons:
- S3 networking latency
- Can GDAL and all the requirements actually fit?
- Cold start up time adds to request time
- Lambda memory seems to max out at 1.5 GB, not sure how limiting that would be for certain queries given other system memory requirements
- Dominant language
- Python
- Stars
- 6
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from azavea/simple-raster-processing
-
Investigate BLASOpen
Difficulty 5/5 Over a week Newbie friendliness 20/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
azavea/simple-raster-processing#18 · 2 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
azavea/simple-raster-processing#10 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
All issues in azavea/simple-raster-processing
Similar issues
-
namespace operations
Difficulty 1/5 Under an hour Newbie friendliness 72/100
EclipseFdn/open-vsx.org#14043 ·
Maintainers usually reply within 1 day
-
netbox status: needs triage type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
netbox-community/netbox#23376 ·
Maintainers usually reply within 1 day
-
feedback simulation workshop
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
githubnext/gh-aw-workshop#4455 ·
Maintainers usually reply within 1 day
-
Triage 🩺
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
[BUG] Container scenario crashes without expected_recovery_time, kube DNS example uses retry_waitOpenneeds-triage
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
krkn-chaos/krkn#1627 · 1 comment ·
Maintainers usually reply within 1 day