5090 Laptop 24G + 96 GB RAM swift-iq3_xxs和qwen-iq3_s 524K 是最优档：decode 完全不掉
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- cpp
- Domain
- machine-learning, performance
Research direction
Start by reviewing the issue’s benchmark images and the reported Qwen settings: --kv q4_0 appears slower than int8, despite using less memory. Reproduce the comparison on the stated 5090 laptop setup and investigate the KV-cache quantization path; the issue does not name files or tests. Done means confirming the reported performance difference and identifying its cause.
Written by the indexing model from the issue text.
Description
重要的发现：qwen 的 --kv q4_0 是负收益，换成 int8 直接快 12%
引擎 help 写 --kv q4_0 是"half of int8's memory"，看着该用。但实测反了：
- Dominant language
- C++
- Stars
- 11.6k
- Forks
- 1k
- Avg merge
- 7h 46m
- Merged PRs (30d)
- 30
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Niko1221/Strata
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
Niko1221/Strata#1248 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
bug iOS 🍎 ui/ux
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
MerginMaps/mobile#4744 ·
Maintainers usually reply within 1 day
-
Component: Ruby Type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 3 days
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
kokkos/kokkos-kernels#3328 ·
Maintainers usually reply within 1 day
-
bug needs triage tcp
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
project-chip/connectedhomeip#74644 ·
Maintainers usually reply within 1 day