ShishirPatil/gorilla

[Apibench] Resume interrupted LLM generations from last generation

開放

#484 建立於 2024年6月24日

 (1 則留言) (0 個反應) (0 位負責人)Python (1,398 個分叉)batch import
apibench-datagood first issue

倉庫指標

星標
 (13,007 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

Describe the issue When we try to launch LLM generations on APIBench, if the generation is interrupted midway, currently the partial file is deleted and restart generation from start. What we should instead do is check where the generation stopped and resume from the last generated index. This is specially critical for large models where generation can take >1 hours.

ID datapoint 2. Provider: This affects all 3: TorchHub/HuggingFace/PyTorch Hub

File that needs to be edited: https://github.com/ShishirPatil/gorilla/blob/main/eval/get_llm_responses.py

貢獻者指南