buggood first issue
Repository metrics
- Stars
- (38,959 個のスター)
- PR merge metrics
- (PR metrics pending)
説明
main branch
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --max-gpu-memory 7Gib --load-8bit
...FastChat/fastchat/model/compression.py:110 in load_compress_model │
│ │
│ 107 │ │ config = AutoConfig.from_pretrained( │
│ 108 │ │ │ model_path, low_cpu_mem_usage=True, torch_dtype=torch_dtype │
│ 109 │ │ ) │
│ ❱ 110 │ │ model = AutoModelForCausalLM.from_config(config) │
│ 111 │ │ linear_weights = get_compressed_list(model) │
│ 112 │ │
│ 113 │ compressed_state_dict = {}
......
ValueError: Unrecognized configuration class <class
'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel:
AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig,
BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig,
CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig,
ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig,
GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig,
MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig,
ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig,
RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig,
TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig,
XLNetConfig, XmodConfig.
I tried to load the model with model = AutoModelForSeq2SeqLM.from_pretrained(model_path, low_cpu_mem_usage=True), it works ok. And then it crash when I input some text.