Reliability monitoring - e2e testing of GitHub workflow

Reliability monitoring - e2e testing of GitHub workflow

Context

Since moving the Communtity Cluster from GKE to DigitalOcean the reliability has dropped dramatically due to issues with the Cilium networking driver 1.4.0. A separate action will be required for this, but in the meantime it highlights a lack of e2e monitoring for the GitHub workflow end-to-end.

Task

Create a Golang function that will:

• clone the repo (it will be a Go function)
• modify the handler's response to include a specific string such as the current date/time stamp
• do a git push (using a secret)
• call the function in a loop with a generous timeout until the response contains the new string
• call into a slack webhook if the string is incorrect or not 200 at the end

Constraints

• The repo in question should be cloned with the go-github library
• This should be an OpenFaaS function
• Use /tmp/ as a writable area
• For write access use the GitHub App approach as per alexellis/derek and openfaas/openfaas-cloud

Suggestions and input is welcome