buggood first issue
Repository metrics
- Stars
- (38,959 stars)
- PR merge metrics
- (30d に merged PR はありません)
説明
Hi, I'm trying to use fastchat-t5-3b-v1.0 on macOS following the instructions in the README.
Simply run the line below to start chatting. It will automatically download the weights from a Hugging Face repo.
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0
When I do this, I get the error "Torch not compiled with CUDA enabled".
I added --device mps --load-8bit to run it as
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --device mps --load-8bit
And that gives a different error: "Unrecognized configuration class <class 'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel: AutoModelForCausalLM".
Is this usable without CUDA?