Any plans to add AutoGPTQ as a gptq load option?

描述

贡献者指南

技术栈

pythonpytorch

领域

machine learningai

议题类型

feature

难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。

4

预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。

1-2 days

活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。

stale

清晰度议题是否清楚说明期望改动、验收标准和下一步。

mostly clear

前置要求

PythonPyTorchFamiliarity with model loading

新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。

35

研究方向

Examine the FastChat codebase to understand how model load options are defined, particularly in the model loading module. Refer to the AutoGPTQ library (https://github.com/PanQiWei/AutoGPTQ) and the example implementation in text generation webui (linked in the issue). Investigate if there is an existing integration pattern for quantization methods. Consider creating a new loader class similar to existing ones, and test with a local model.