Add GenAI-Perf Benchmark tool
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning, performance
Research direction
The issue names GenAI-Perf but does not identify any files, tests, or entry points. First determine where deployment backends and performance tooling are organized, then define the integration scope and measurable inference benchmarks; done should mean GenAI-Perf support is implemented and its results can be used to compare deployment options.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
Measuring the inference performance is important and we need GenAI-Perf tool support to do it. This can be used to find the best inference option for deployment.
- Dominant language
- Python
- Stars
- 43
- Forks
- 19
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/Export-Deploy
-
nvidia-runners
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
NVIDIA-NeMo/Export-Deploy#759 ·
-
Update ED to use Dynamic Inference Engine from MCoreMay be free again @oyilmaz-nvidia claimed this 149 days ago, and no pull request is open. Open
NVIDIA-NeMo/Export-Deploy#693 · 1 assignee ·
-
Incorrect handling of reasoning traces for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 in MBridge formatMay be free again @athitten claimed this 149 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#682 · 1 comment · 1 assignee ·
-
Server returns empty answer when too many tokens requestedMay be free again @athitten claimed this 188 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#652 · 1 assignee ·
-
VLM endpoint fails when there are no images in the requestMay be free again @meatybobby claimed this 220 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/Export-Deploy#632 · 1 assignee ·
All issues in NVIDIA-NeMo/Export-Deploy
Similar issues
-
tunneld on Windows: Wi-Fi devices are not discovered because uvicorn runs on a ProactorEventLoopOpen
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
doronz88/pymobiledevice3#2051 ·
Maintainers usually reply within 1 day
-
Remove DatacircleOpen
Difficulty 1/5 Under an hour Newbie friendliness 90/100
collabnix/awesome-mcp-lists#230 ·
Maintainers usually reply within 9 days
-
Groc schedules with out of range values pass config validation and fail later when the scheduler is builtPossibly taken @auraseb claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
autowarefoundation/autoware_carla_scenario#342 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
ianktoo/project-monitor#14 ·