Evaluation of HF Models with LIDA

What

Local models (e.g. LLAMA based models available via HuggingFace in the 7B or 13B size classes) offer multiple benefits (e.g., can be finetuned/adapted, run locally etc). While LIDA has been mostly tested with OpenAI models, more work is needed to test workflows and performance for HF models.

Work Items

• Test a set of local HF models either directly with llmx or with LIDA, systematically document bugs and suggest fixes via PRs