Support quantized m-bridge checkpoint for vLLM export
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by tracing the vLLM export path and its existing HF export handling, then compare how m-bridge represents quantized modelopt checkpoints. Done means quantized m-bridge checkpoints can be exported through vLLM, with coverage for the supported path.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
Currently vLLM export uses HF export and m-bridge doesn't have any HF export for quantized models with modelopt. This should be supported as well.
- Dominant language
- Python
- Stars
- 43
- Forks
- 19
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/Export-Deploy
-
nvidia-runners
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
NVIDIA-NeMo/Export-Deploy#759 ·
-
Update ED to use Dynamic Inference Engine from MCoreMay be free again @oyilmaz-nvidia claimed this 151 days ago, and no pull request is open. Open
NVIDIA-NeMo/Export-Deploy#693 · 1 assignee ·
-
Incorrect handling of reasoning traces for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 in MBridge formatMay be free again @athitten claimed this 151 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#682 · 1 comment · 1 assignee ·
-
Server returns empty answer when too many tokens requestedMay be free again @athitten claimed this 190 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#652 · 1 assignee ·
-
VLM endpoint fails when there are no images in the requestMay be free again @meatybobby claimed this 222 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#632 · 1 assignee ·
All issues in NVIDIA-NeMo/Export-Deploy
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Maintainers usually reply within 1 day
-
good first issue module-baseline needs triage ux
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
darnitdevorg/darnit#650 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
nyjc-computing/campus#938 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
nf-osi/nf-metadata-dictionary#1027 ·
Maintainers usually reply within 1 day
-
bug documentation
Difficulty 1/5 Under an hour Newbie friendliness 92/100
qatrackplus/qatrackplus#913 ·
Maintainers usually reply within 3 days