speed up phi-3 inference?
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- android, cpp
- Domain
- machine-learning, mobile, performance
Research direction
Start with the phi-3 Android example and determine which execution path it currently uses. Compare the available GPU and Qualcomm HTP options, then establish whether accelerated inference can be demonstrated with a measurable speed improvement.
Written by the indexing model from the issue text.
Description
Hi, I tried your phi-3 example on Android. I wonder if it can run on GPU/Qualcomm HTP to further increase speed? Currently I suppose it only uses CPU.
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 418
- PR merge metrics
- No merged PRs in 30d
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/onnxruntime-inference-examples
-
Replace deprecated light-server with lite-server for onnxruntime-webPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Minor: typoPossibly taken @Hashim1999164 claimed this 67 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 72/100
microsoft/onnxruntime-inference-examples#124 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 38/100
-
Package "onnxruntime-react-native" does not contain a valid config pluginPossibly taken @reginaldalfret claimed this 25 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 42/100
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
All issues in microsoft/onnxruntime-inference-examples
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 91/100
FujiNetWIFI/fujinet-firmware#2045 ·
Maintainers usually reply within 1 day
-
(S1 - Need confirmation)
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
CleverRaven/Cataclysm-DDA#89159 ·
Maintainers usually reply within 1 day
-
area:libnvmpi bug P3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
gjrtimmer/jetson-nvmpi#33 ·
-
bug(wasi_crypto): ArrayOutput::pull reports the wrong all-pulled flagPossibly taken @Yashbhu claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
bug question
Difficulty 1/5 Under an hour Newbie friendliness 62/100