buggood first issue
倉庫指標
- Star
- (38,959 star)
- PR 合併指標
- (30 天內沒有已合併 PR)
描述
Hi, I'm trying to use fastchat-t5-3b-v1.0 on macOS following the instructions in the README.
Simply run the line below to start chatting. It will automatically download the weights from a Hugging Face repo.
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0
When I do this, I get the error "Torch not compiled with CUDA enabled".
I added --device mps --load-8bit to run it as
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --device mps --load-8bit
And that gives a different error: "Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForCausalLM".
Is this usable without CUDA?