awslabs/benchmark-ai
[Executor] Dangling resources on failed kubectl apply
Ouverte
#943 ouverte le 14 oct. 2019
buggood first issueservice:executor
Métriques du dépôt
- Stars
- (19 étoiles)
- Métriques de merge PR
- (Aucune PR mergée en 30 j)
Description
The executor creates the kubernetes resources by executing a kubectl apply against a generated kubernetes resource file. Sometimes, these files can contain multiple resource definitions (Jobs, Pods, Services, Roles, etc.). This means that, while the command itself can fail (exit with non-zero), some of the (valid) kuberenetes resources would still be created.
We should issue a kubectl delete in case there is a failure. To ensure we don't leave any dangling resources in case of any failure.