Find the fastest inference options among vLLM, TensorRT-LLM, and M-Bridge
@Saurav-Gupta-9741 is already working on this.
Since Jun 19, 2026.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- machine-learning
Research direction
No files, tests, or entry points are named. Start by examining how vLLM, TensorRT-LLM, and M-Bridge inference options are represented in Export-Deploy, then clarify how speed should be measured and compared. Done should mean the feature can identify the fastest supported option using agreed criteria.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
ED has many inference options such as vLLM, TensorRT-LLM, and M-Bridge and a feature that finds the fastest inference option is needed.
- Dominant language
- Python
- Stars
- 43
- Forks
- 18
- Avg merge
- 7h 17m
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/Export-Deploy
-
nvidia-runners
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
NVIDIA-NeMo/Export-Deploy#759 ·
-
Update ED to use Dynamic Inference Engine from MCoreMay be free again @oyilmaz-nvidia claimed this 143 days ago, and no pull request is open. Open
NVIDIA-NeMo/Export-Deploy#693 · 1 assignee ·
-
Incorrect handling of reasoning traces for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 in MBridge formatMay be free again @athitten claimed this 143 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#682 · 1 comment · 1 assignee ·
-
Server returns empty answer when too many tokens requestedMay be free again @athitten claimed this 183 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#652 · 1 assignee ·
-
VLM endpoint fails when there are no images in the requestMay be free again @meatybobby claimed this 214 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#632 · 1 assignee ·
All issues in NVIDIA-NeMo/Export-Deploy
Similar issues
-
bug difficulty:2 good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
MakazhanAlpamys/Soup#1865 ·
Maintainers usually reply within 1 day
-
area: agents area: docs good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
openapi/gknowledge#23 ·
-
useless-suppression reported for deprecated-* messages that should be exemptPossibly taken @adityasinha-ghub claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
pylint-dev/pylint#11656 ·
Maintainers usually reply within 1 day
-
area:provisioning effort:S priority:P2 provider:local-libvirt risk:daytime-only status:in-progress type:bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
randomparity/kdive#3203 · 2 comments ·
Maintainers usually reply within 1 day
-
get() for StrEnumPossibly taken @FeJo5952 claimed this today. Opengood first issue
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
collective/icalendar#1921 · 1 comment ·
Maintainers usually reply within 1 day