是否有计划优化GPU上的推理加速
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- cpp
- Domain
- machine-learning, performance
Research direction
Start by reviewing the existing GPTQ quantization-layer kernel implementation and GPU inference path; the issue does not name files or tests. Benchmark quantized inference against the current implementation and define done as a measurable GPU speedup rather than a slowdown.
Written by the indexing model from the issue text.
Description
目前社区LLM采用主流GPTQ量化之后，量化层的kernel实现基本是负向优化，是否有计划支持GPU上量化后的模型推理加速。
- Dominant language
- C++
- Stars
- 752
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MegEngine/InferLLM
-
SG2042 服务器 make 出错 Open
Difficulty 4/5 3-5 days Newbie friendliness 30/100
-
无法加载模型bin或者guff模型 Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
有打算支持qwen吗 Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
-
这个有windows的部署教程吗 Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
All issues in MegEngine/InferLLM
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·