s3_handler doesn't handle delete events
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 52/100
Research direction
Read steps/handlers/s3_handler.py, starting with handle and _extract_data, and inspect how the S3 event data identifies delete events or delete markers before get_object is called. Done means S3 delete notifications can be forwarded without attempting to retrieve the deleted object or raising NoSuchKey.
Written by the indexing model from the issue text.
Description
Expected Behavior
I want to forward logs from s3 delete event
s3_handler._extract_data shouldn't try to retreive an object if it has delete marker or even is of type delete
Actual Behavior
following method in s3_handler fails as it tries to access deleted object.
def _extract_data(self):
s3_client = self._get_s3_client()
response = s3_client.get_object(
Bucket=self.data_store.bucket, Key=self.data_store.key
)
body = response.get("Body")
self.data_store.data = body.read()
Steps to Reproduce the Problem
- Set up s3 bucket notification to trigger templated lambda upon object delete
Specifications
- Datadog CloudFormation template version:
Stacktrace
[ERROR] NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
Traceback (most recent call last):
File "/var/task/datadog_lambda/wrapper.py", line 239, in __call__
"""Executes when the wrapped function gets called"""
File "/var/task/ddtrace/contrib/internal/aws_lambda/patch.py", line 119, in __call__
self.response = self.func(*args, **kwargs)
File "/var/task/datadog_lambda/wrapper.py", line 242, in __call__
self.response = self.func(event, context, **kwargs)
File "/var/task/lambda_function.py", line 84, in datadog_forwarder
parsed = parse(event, context, cache_layer)
File "/var/task/steps/parsing.py", line 56, in parse
return normalize_events(events, metadata)
File "/var/task/steps/parsing.py", line 125, in normalize_events
for event in events:
File "/var/task/steps/handlers/s3_handler.py", line 64, in handle
self._extract_data()
File "/var/task/steps/handlers/s3_handler.py", line 118, in _extract_data
response = s3_client.get_object(
File "/var/lang/lib/python3.12/site-packages/botocore/client.py", line 569, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/ddtrace/contrib/trace_utils.py", line 336, in wrapper
return func(mod, pin, wrapped, instance, args, kwargs)
File "/var/task/ddtrace/contrib/internal/botocore/patch.py", line 200, in patched_api_call
return patching_fn(
File "/var/task/ddtrace/contrib/internal/botocore/patch.py", line 260, in patched_api_call_fallback
result = original_func(*args, **kwargs)
File "/var/lang/lib/python3.12/site-packages/botocore/client.py", line 1023, in _make_api_call
raise error_class(parsed_response, operation_name)
- Dominant language
- Python
- Stars
- 47
- Forks
- 58
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 8
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/cloudformation-template
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
DataDog/cloudformation-template#342 · 1 reaction ·
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
DataDog/cloudformation-template#174 · 1 comment ·
-
When will the templates in aws quickstart be updated to use the latest version of python (3.13) Open
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
All issues in DataDog/cloudformation-template
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100