buggood first issue
Metriche repository
- Star
- (38.959 star)
- Metriche merge PR
- (Nessuna PR mergiata in 30 g)
Descrizione
main branch
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --max-gpu-memory 7Gib --load-8bit
...FastChat/fastchat/model/compression.py:110 in load_compress_model │
│ │
│ 107 │ │ config = AutoConfig.from_pretrained( │
│ 108 │ │ │ model_path, low_cpu_mem_usage=True, torch_dtype=torch_dtype │
│ 109 │ │ ) │
│ ❱ 110 │ │ model = AutoModelForCausalLM.from_config(config) │
│ 111 │ │ linear_weights = get_compressed_list(model) │
│ 112 │ │
│ 113 │ compressed_state_dict = {}
......
ValueError: Unrecognized configuration class <class
'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel:
AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig,
BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig,
CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig,
ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig,
GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig,
MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig,
ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig,
RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig,
TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig,
XLNetConfig, XmodConfig.
I tried to load the model with model = AutoModelForSeq2SeqLM.from_pretrained(model_path, low_cpu_mem_usage=True), it works ok. And then it crash when I input some text.