awslabs/benchmark-ai

[Executor] Dangling resources on failed kubectl apply

開放

#943 建立於 2019年10月14日

 (0 則留言) (0 個反應) (0 位負責人)Python (6 個分叉)auto 404
buggood first issueservice:executor

倉庫指標

星標
 (19 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

The executor creates the kubernetes resources by executing a kubectl apply against a generated kubernetes resource file. Sometimes, these files can contain multiple resource definitions (Jobs, Pods, Services, Roles, etc.). This means that, while the command itself can fail (exit with non-zero), some of the (valid) kuberenetes resources would still be created.

We should issue a kubectl delete in case there is a failure. To ensure we don't leave any dangling resources in case of any failure.

貢獻者指南