Thread wakening may be bottom neck for large core systems
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- cpp
- Domain
- backend, performance
Research direction
Start with the Tracy traces described in the issue and investigate the worker-wakening path around thread::yield(). Reproduce the delayed task pickup on SG2042, then measure whether a busier polling strategy improves throughput without preventing the workers from becoming idle while waiting for user input.
Written by the indexing model from the issue text.
Description
Result: boost from 3.6~8token/s to 4.2~4.3token/s on SG2042.
Analyze: I added tracy to trace execution in details. I observed that worker wakening is problematic. There are some workers picked up task after the completion of other workers.
That means the execution time is as twice as the expected sometimes. I guess that it is caused by thread::yield(), which switched out the busy waiting loop.
I am thinking about to make a higher/busier performance poll, while being able to switch to an idle state when waiting for user input.
- Dominant language
- C++
- Stars
- 752
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MegEngine/InferLLM
-
SG2042 服务器 make 出错Open
Difficulty 4/5 3-5 days Newbie friendliness 30/100
-
无法加载模型bin或者guff模型Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
有打算支持qwen吗Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
-
这个有windows的部署教程吗Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
All issues in MegEngine/InferLLM
Similar issues
-
bug chart-audit
Difficulty 1/5 Under an hour Newbie friendliness 92/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
godotengine/godot#124120 ·
Maintainers usually reply within 1 day
-
HasBacktrace Priority-Critical
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
azerothcore/azerothcore-wotlk#27921 ·
Maintainers usually reply within 1 day
-
area/ysql kind/bug priority/medium
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
yugabyte/yugabyte-db#34584 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day