buggood first issue
仓库指标
- 星标
- (38,959 个星标)
- PR 合并指标
- (PR 指标待抓取)
描述
Hi, I'm trying to use fastchat-t5-3b-v1.0 on macOS following the instructions in the README.
Simply run the line below to start chatting. It will automatically download the weights from a Hugging Face repo.
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0
When I do this, I get the error "Torch not compiled with CUDA enabled".
I added --device mps --load-8bit to run it as
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --device mps --load-8bit
And that gives a different error: "Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForCausalLM".
Is this usable without CUDA?