GoogleCloudPlatform/terraform-splunk-log-export

replaying deadletter with replay udf and includePubsubMessage=true doesn't work

Aperta

#38 aperta il 3 mar 2023

 (2 commenti) (0 reazioni) (0 assegnatari)HCL (32 fork)auto 404
bugdocumentationgood first issue

Metriche repository

Star
 (46 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

The default configured behavior of the log export pipeline is to force includePubsubMessage to true[1].

When used in combination with sample Dataflow UDF message replay Javascript[2], replayed messages from the deadletter subscription won't be handled correctly. As messages enter the original/main log subscription, they will be re-wrapped in the data key again. You will end up with strange logs like this in Splunk:

{
  "data": {
    "time": 1677812444044,
    "event": "{\"data\":{\"insertId\":\"sv1hmye6bz6p\",\"logName\":\"projects/redacted-project/logs/[cloudaudit.googleapis.com](http://cloudaudit.googleapis.com/)%2Fdata_access\",\"protoPayload\":{\"@type\":\"[type.googleapis.com/google.cloud.audit.AuditLog\](http://type.googleapis.com/google.cloud.audit.AuditLog%5C)",\"authenticationInfo\":{\"principalEmail\":\"[redacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com](mailto:redacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com)\",\"principalSubject\":\"[serviceAccount:redacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com](mailto:serviceAccount%3Aredacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com)\",\"serviceAccountKeyName\":\"//[iam.googleapis.com/projects/redacted-project/serviceAccounts/redacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com/keys/a49e3382174e3775dec6fd4e5b593dbfed8c1ecc\](http://iam.googleapis.com/projects/redacted-project/serviceAccounts/redacteduser-sfx-scraper@redacted-project.iam.gserviceaccount.com/keys/a49e3382174e3775dec6fd4e5b593dbfed8c1ecc%5C)"},\"authorizationInfo\":[{\"granted\":true,\"permission\":\"monitoring.timeSeries.list\",\"resource\":\"50701599922\",\"resourceAttributes\":{}}],\"methodName\":\"google.monitoring.v3.MetricService.ListTimeSeries\",\"request\":{\"@type\":\"[type.googleapis.com/google.monitoring.v3.ListTimeSeriesRequest\](http://type.googleapis.com/google.monitoring.v3.ListTimeSeriesRequest%5C)",\"filter\":\"metric.type = \\\"[kubernetes.io/node/cpu/allocatable_utilization\\\](http://kubernetes.io/node/cpu/allocatable_utilization%5C%5C%5C)"\",\"name\":\"projects/redacted-project\",\"pageSize\":10000},\"requestMetadata\":{\"callerIp\":\"44.230.82.104\",\"callerSuppliedUserAgent\":\"grpc-java-netty/1.51.1,gzip(gfe)\",\"destinationAttributes\":{},\"requestAttributes\":{\"auth\":{},\"time\":\"2023-03-03T03:00:44.050942344Z\"}},\"resourceName\":\"projects/redacted-project\",\"serviceName\":\"[monitoring.googleapis.com](http://monitoring.googleapis.com/)\"},\"receiveTimestamp\":\"2023-03-03T03:00:44.174642381Z\",\"resource\":{\"labels\":{\"method\":\"google.monitoring.v3.MetricService.ListTimeSeries\",\"project_id\":\"redacted-project\",\"service\":\"[monitoring.googleapis.com](http://monitoring.googleapis.com/)\"},\"type\":\"audited_resource\"},\"severity\":\"INFO\",\"timestamp\":\"2023-03-03T03:00:44.044614574Z\"},\"attributes\":{\"[logging.googleapis.com/timestamp\](http://logging.googleapis.com/timestamp%5C)":\"2023-03-03T03:00:44.044614574Z\"},\"messageId\":\"7058887762594632\",\"delivery_attempt\":1}"
  },
  "attributes": {
    "errorMessage": "Splunk write status code: 403",
    "timestamp": "2023-03-03 03:00:34.000000"
  },
  "messageId": "7058953036869866",
  "delivery_attempt": 1
}

[1] - https://github.com/GoogleCloudPlatform/terraform-splunk-log-export/blob/main/main.tf#L85 [2] - https://storage.googleapis.com/splk-public/js/dataflow_udf_messages_replay.js

Guida contributor