Quantize M-Bridge and HF checkpoints during In-Framework Deployment
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- huggingface, python
- Domain
- machine-learning
Research direction
The issue names no files, tests, or entry points. Start by locating the in-framework deployment paths for M-Bridge and Hugging Face checkpoints, then clarify which quantization modes are required. Done should mean users can quantize both checkpoint types during in-framework deployment.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
During in-framework deployment, there is no option to quantize the checkpoint. Some users asked for this feature.
- Dominant language
- Python
- Stars
- 43
- Forks
- 18
- Avg merge
- 7h 17m
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/Export-Deploy
-
nvidia-runners
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
NVIDIA-NeMo/Export-Deploy#759 ·
-
Update ED to use Dynamic Inference Engine from MCoreMay be free again @oyilmaz-nvidia claimed this 143 days ago, and no pull request is open. Open
NVIDIA-NeMo/Export-Deploy#693 · 1 assignee ·
-
Incorrect handling of reasoning traces for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 in MBridge formatMay be free again @athitten claimed this 143 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#682 · 1 comment · 1 assignee ·
-
Server returns empty answer when too many tokens requestedMay be free again @athitten claimed this 182 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#652 · 1 assignee ·
-
VLM endpoint fails when there are no images in the requestMay be free again @meatybobby claimed this 214 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#632 · 1 assignee ·
All issues in NVIDIA-NeMo/Export-Deploy
Similar issues
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 88/100
openwisp/openwisp-utils#777 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
google-research/weatherbenchX#310 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/peft#3955 ·
Maintainers usually reply within 1 day
-
author-page correction
Difficulty 2/5 1-3 hours Newbie friendliness 60/100
acl-org/acl-anthology#10313 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
AOSSIE-Org/Gitcord-GithubDiscordBot#147 · 1 comment ·
Maintainers usually reply within 1 day