huggingface/lighteval

[FT] LiteLLM concurrency parameters hard-coded

开放

#567 创建于 2025年2月16日

 (1 条评论) (0 个反应) (0 位负责人)Python (514 个派生)auto 404
featuregood first issuehelp wanted

仓库指标

星标
 (2,496 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

I noticed that for LiteLLM the concurrency is hard-coded for an arbitrary number (20). Well, a lot of the request behavior is hard coded, but the concurrency is the one that is probably least likely to work well as a one size fits all.

If everything is going through the YAML file, maybe it can be expanded, although the YAML file is another ball of wax (makes automated runs much harder when you can't just pass in parameters via a call)

贡献者指南