--load-8bit not compatiable for fastchat-t5-3b-v1.0

main branch

python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --max-gpu-memory 7Gib --load-8bit

 ...FastChat/fastchat/model/compression.py:110 in load_compress_model            │
│                                                                                                  │
│   107 │   │   config = AutoConfig.from_pretrained(                                               │
│   108 │   │   │   model_path, low_cpu_mem_usage=True, torch_dtype=torch_dtype                    │
│   109 │   │   )                                                                                  │
│ ❱ 110 │   │   model = AutoModelForCausalLM.from_config(config)                                   │
│   111 │   │   linear_weights = get_compressed_list(model)                                        │
│   112 │                                                                                          │
│   113 │   compressed_state_dict = {}

......

ValueError: Unrecognized configuration class <class
'transformers.models.t5.configuration_t5.T5Config'> for this kind of AutoModel:
AutoModelForCausalLM.
Model type should be one of BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig,
BigBirdPegasusConfig, BioGptConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig,
CamembertConfig, CodeGenConfig, CpmAntConfig, CTRLConfig, Data2VecTextConfig, ElectraConfig,
ErnieConfig, GitConfig, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig,
GPTNeoXJapaneseConfig, GPTJConfig, LlamaConfig, MarianConfig, MBartConfig, MegaConfig,
MegatronBertConfig, MvpConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PLBartConfig,
ProphetNetConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig,
RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, Speech2Text2Config, TransfoXLConfig,
TrOCRConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig,
XLNetConfig, XmodConfig.

I tried to load the model with model = AutoModelForSeq2SeqLM.from_pretrained(model_path, low_cpu_mem_usage=True), it works ok. And then it crash when I input some text.