mayocream/koharu

Shared gpu Memory

Open

#638 opened on May 1, 2026

View on GitHub
 (3 comments) (0 reactions) (0 assignees)Rust (257 forks)github user discovery
help wanted

Repository metrics

Stars
 (4,568 stars)
PR merge metrics
 (PR metrics pending)

Description

does koharu can't utilize shared gpu memory? because im using sugoi 14b ultra models and sometimes it just stop translating after dedicated memory is full

my system is using rtx 3060 12Gb vram intel core i5 10400f and 64Gb ram. so i have about 32gb shared memory

sugoi 14b ultra model is really nice at translating, thats why i like it even it is slower than vntl-llama3-8b-v2.

Contributor guide

Research direction
Investigate how koharu handles GPU memory allocation, especially shared memory. Check if there is a configuration option for memory usage or if the model loading can be adjusted to use shared memory when dedicated memory is full. Look at dependencies like `candle` or `llama cpp rs` for memory management.
Tech stack
rust
Domain
desktopmachine learning
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
GitRust
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65