[Upstream]: SGlang inference with AR kernels
Maintainers usually reply within 1 day
@luoyu-intel is already working on this.
Since Mar 20, 2026.
Assessment
This issue has not been assessed yet.
Description
Feature Description
Load the AR quantized model and inference with AR kernels.
Motivation and Use Case
Support AR in the mainstream inference framework
Alternatives Considered
No response
Definition of Done
No response
Additional Context
No response
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 193
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 52
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from intel/auto-round
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
intel/auto-round#2525 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 44/100
intel/auto-round#2532 ·
Maintainers usually reply within 1 day
-
[Bug]: FP8 static KV cache accuracy issuePossibly taken @xin3he claimed this 7 days ago. Openbug
intel/auto-round#2518 · 1 comment · 1 reaction · 1 assignee ·
Maintainers usually reply within 1 day
-
[Feature]: split special_model_handler to a folderPossibly taken @Cross2pro claimed this today. Openenhancement
Difficulty 3/5 Half a day Newbie friendliness 42/100
intel/auto-round#2514 · 4 comments ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 20/100
intel/auto-round#2507 ·
Maintainers usually reply within 1 day
All issues in intel/auto-round
Similar issues
-
bug P1
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Toloka/tolokaforge#1947 ·
Maintainers usually reply within 1 day
-
SageMaker provider: streaming fails with NoStreamDataError on DJL Serving (LMI) endpointsPossibly taken @balajiaru claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
UKGovernmentBEIS/inspect_ai#6127 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
pydot/pydot#601 · 4 comments ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
letsencrypt/cp-cps#361 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100