buggood first issue
Métriques du dépôt
- Stars
- (38 959 stars)
- Métriques de merge PR
- (Aucune PR mergée en 30 j)
Description
main branch
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --max-gpu-memory 7Gib --load-8bit
...FastChat/fastchat/model/compression.py:110 in load_compress_model │
│ │
│ 107 │ │ config = AutoConfig.from_pretrained( │
│ 108 │ │ │ model_path, low_cpu_mem_usage=True, torch_dtype=torch_dtype │
│ 109 │ │ ) │
│ ❱ 110 │ │ model = AutoModelForCausalLM.from_config(config) │
│ 111 │ │ linear_weights = get_compressed_list(model) │
│ 112 │ │
│ 113 │ compressed_state_dict = {}
......
ValueError: Unrecognized configuration class <class
'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel:
AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig,
BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig,
CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig,
ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig,
GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig,
MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig,
ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig,
RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig,
TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig,
XLNetConfig, XmodConfig.
I tried to load the model with model = AutoModelForSeq2SeqLM.from_pretrained(model_path, low_cpu_mem_usage=True), it works ok. And then it crash when I input some text.