awslabs/benchmark-ai

[Executor] Refactor to use templating engine

Aperta

#942 aperta il 14 ott 2019

 (1 commento) (1 reazione) (0 assegnatari)Python (6 fork)auto 404
enhancementgood first issueservice:executor

Metriche repository

Star
 (19 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

One of the main functions of the executor is to transpile the toml to a set of kubernetes resource definition in yaml. Currently, we use some home brew for this. A mixture of python templates and object manipulation.

I think it would make sense to consolidate all the required template information, i.e. descriptor, data sources, models, scripts, executor configuration, fetcher event, etc. into a context object that can back the template.

Then, I would substitute the transpiler for a templating engine. We could either use Jinja2 or even just ship helm in the executors environment then issue a helm template command to render the resource files based on helm templates.

Templating engines support the addition of logic, which means the resource files could be solely generated from the backing context object and the templates themselves. Thereby, we can just remove our home made solution, reducing the maintenance burden.

We have pretty good transpiler toml -> resource tests, which can really help in this refactoring effort.

Guida contributor