lm-sys/FastChat

--load-8bit not compatiable for fastchat-t5-3b-v1.0

Open

#1,099 opened on 2023年5月9日

GitHub で見る
 (0 comments) (0 reactions) (1 assignee)Python (4,736 forks)batch import
buggood first issue

Repository metrics

Stars
 (38,959 stars)
PR merge metrics
 (30d に merged PR はありません)

説明

main branch

python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --max-gpu-memory 7Gib --load-8bit

 ...FastChat/fastchat/model/compression.py:110 in load_compress_model            │
│                                                                                                  │
│   107 │   │   config = AutoConfig.from_pretrained(                                               │
│   108 │   │   │   model_path, low_cpu_mem_usage=True, torch_dtype=torch_dtype                    │
│   109 │   │   )                                                                                  │
│ ❱ 110 │   │   model = AutoModelForCausalLM.from_config(config)                                   │
│   111 │   │   linear_weights = get_compressed_list(model)                                        │
│   112 │                                                                                          │
│   113 │   compressed_state_dict = {}

......

ValueError: Unrecognized configuration class <class
'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel:
AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig,
BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig,
CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig,
ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig,
GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig,
MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig,
ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig,
RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig,
TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig,
XLNetConfig, XmodConfig.

I tried to load the model with model = AutoModelForSeq2SeqLM.from_pretrained(model_path, low_cpu_mem_usage=True), it works ok. And then it crash when I input some text.

コントリビューターガイド