Text appears and disappears when POD is launched with models larger than 10B
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 20/100
Research direction
No source file, test, or entry point is identified. Start by reproducing the behavior with a model over 10B and inspect the attached Triton pod error, then determine why the generated text disappears and verify that the output remains visible after the fix.
Written by the indexing model from the issue text.
Description
hello.
As the title says, when I use a model over 10B and reference it, the text comes up and then disappears.
I would appreciate your opinion on what could be the cause and how to fix it.
Thank you.
I'm also attaching the triton pod related error.
Received stop request for requestId 7796140 but it's not active (might be completed already).
- Dominant language
- Jupyter Notebook
- Stars
- 4.2k
- Forks
- 1.1k
- Avg merge
- 10h 15m
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/GenerativeAIExamples
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
NVIDIA/GenerativeAIExamples#361 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/GenerativeAIExamples#299 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
NVIDIA/GenerativeAIExamples#437 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 45/100
NVIDIA/GenerativeAIExamples#400 ·
-
Difficulty 5/5 Over a week Newbie friendliness 10/100
NVIDIA/GenerativeAIExamples#399 ·
All issues in NVIDIA/GenerativeAIExamples
Similar issues
-
ai-observability bug team/ai-observability
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
dair-ai/Prompt-Engineering-Guide#810 · 1 comment ·
-
[addtool] DeepZero Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
gwen001/offsectools_www#2055 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
UKGovernmentBEIS/inspect_evals#2503 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
google-gemma/gemma-skills#13 ·