Any plans to add AutoGPTQ as a gptq load option?

Description

Contributor guide

Tech stack

pythonpytorch

Domain

machine learningai

Issue type

feature

DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.

4

Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.

1-2 days

Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.

stale

ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.

mostly clear

Prerequisites

PythonPyTorchFamiliarity with model loading

Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.

35

Research direction

Examine the FastChat codebase to understand how model load options are defined, particularly in the model loading module. Refer to the AutoGPTQ library (https://github.com/PanQiWei/AutoGPTQ) and the example implementation in text generation webui (linked in the issue). Investigate if there is an existing integration pattern for quantization methods. Consider creating a new loader class similar to existing ones, and test with a local model.