awslabs/benchmark-ai

[Executor] Refactor to use templating engine

開放

#942 建立於 2019年10月14日

 (1 則留言) (1 個反應) (0 位負責人)Python (6 個分叉)auto 404
enhancementgood first issueservice:executor

倉庫指標

星標
 (19 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

One of the main functions of the executor is to transpile the toml to a set of kubernetes resource definition in yaml. Currently, we use some home brew for this. A mixture of python templates and object manipulation.

I think it would make sense to consolidate all the required template information, i.e. descriptor, data sources, models, scripts, executor configuration, fetcher event, etc. into a context object that can back the template.

Then, I would substitute the transpiler for a templating engine. We could either use Jinja2 or even just ship helm in the executors environment then issue a helm template command to render the resource files based on helm templates.

Templating engines support the addition of logic, which means the resource files could be solely generated from the backing context object and the templates themselves. Thereby, we can just remove our home made solution, reducing the maintenance burden.

We have pretty good transpiler toml -> resource tests, which can really help in this refactoring effort.

貢獻者指南