serve: 12 GB 카드에서 qwen35moe 기본 ctx가 슬롯당 2048이라 6K 토큰 prompt를 거부
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- rust
- Domain
- backend, machine-learning
Research direction
Start with crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs, especially default_ctx around lines 188–265, and trace how the selected context is divided between two slots. Reproduce with the stated RTX 3060, host memory, and Qwen3.8-35B-A3B Q4_K_M setup, then compare against --parallel 1 or an explicit --ctx-size. Done when flag-free startup accepts prompts around 4K–8K tokens while respecting available memory and slot sizing.
Written by the indexing model from the issue text.
Description
12 GB 카드(RTX 3060) + 호스트 27 GB에서 Qwen3.8-35B-A3B Q4_K_M을 플래그 없이 bloomery-serve로 띄우면 기본 ctx가 합계 4096, 슬롯 2개라 슬롯당 2048입니다. 그래서 6K 토큰 정도의 prompt가 거부됩니다. 플래그 없는 기본값이라면 4K–8K prompt는 받아야 합니다.
- 원인:
general.architecture = qwen35moe파일은 qwen3 seat로 갑니다(crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs). placed 기본값 탐색이 floor 위에서 맞는 ctx를 찾지 못하면 floor인CTX = 4096(167행, 236행)을 돌려줍니다. 그 합계를 슬롯 2개(945행)로 나눠 슬롯당 2048이 됩니다(899행). 슬롯당 최소값은SLOT_CTX_MIN = 8(885행)뿐입니다. - 우회:
--parallel 1또는--ctx-size를 직접 지정. - 고칠 곳:
qwen3.rs의default_ctx(188–265행 부근). 슬롯당 floor를 두거나, room이 모자라면 슬롯 수를 줄이는 쪽이 후보입니다.
메인테이너가 고칩니다. 같은 증상을 겪으셨으면 머신 사양과 함께 댓글로 알려 주세요.
- Dominant language
- Rust
- Stars
- 21
- Forks
- 4
- Avg merge
- 3h 52m
- Merged PRs (30d)
- 134
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from midagedev/bloomery
-
review-bot
Difficulty 2/5 Under an hour Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
hidden_ref: invalid numeric options silently select defaultsPossibly taken @midagedev claimed this today. Openreview-bot
Difficulty 2/5 1-3 hours Newbie friendliness 50/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
Maintainers usually reply within 1 day
-
review-bot
Difficulty 3/5 1-2 days Newbie friendliness 56/100
midagedev/bloomery#214 · 1 comment ·
Maintainers usually reply within 1 day
All issues in midagedev/bloomery
Similar issues
-
agent:triaged bug bughunt pm:bundler priority:p1
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
SocketDev/socket-patch#1573 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
bytecodealliance/rustix#1706 · 1 comment ·
Maintainers usually reply within 11 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
wormhole-foundation/wormhole#5062 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day