Reducing NeMo-Speech working-set memory usage on Windows
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 半天
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- cpp
调研方向
The payload names no source files, so start from the model loading path where the GGUF loader and mmap are mentioned, and check whether the Windows-only memory calls belong in a platform-specific file. The reporter's experiment used the Windows EmptyWorkingSet() API, so confirm first with a maintainer whether an optional trim is wanted at all. Done means an opt-in trim that runs without unloading the model, with the measured working set reported before and after.
由索引模型根据 Issue 内容生成。
描述
Hi, I have been testing NeMo-Speech in Windows on a Nvidia GPU using CUDA. I'm RAM-limited because I have a lot of other software running which was one of the reasons I wanted to offload the speech model to the GPU.
My issue is after loading, nemo-speech.exe was showing about 633 MB of private working set in Windows Task Manager, despite the model weights being loaded onto the GPU. I wondered whether mmap or the GGUF loader's temporary buffers were responsible, but disabling mmap did not resolve the issue. Using VMMap showed substantial committed heap memory, although only a fraction of it was resident in physical RAM.
I experimented with the Windows API EmptyWorkingSet(), which allows Windows to remove pages from a process's working set without terminating the process or releasing its underlying allocations.
The results were:
Initial private working set | ~633 MB
Working set after trimming | ~2.2 MB
After a subsequent transcription | ~50 MB
During continued dictation | ~around 55MB and it settles there.
Private bytes | ~6.38 GB (unchanged)
So I seem to have recovered about 580MB of system RAM without a noticeable downside.
Would it be worth considering an optional Windows-specific working-set trimming mechanism? A one-off trim could substantially reduce the resident RAM footprint without unloading the model, and it can then grow naturally during use.
- 主要语言
- C++
- 星标
- 207
- 派生
- 47
- 平均合并
- 13 小时 10 分钟
- 30 天内合并 PR
- 12
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
NVIDIA/NeMo-Speech.cpp 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 45/100
NVIDIA/NeMo-Speech.cpp#98 ·
维护者通常 1 天内回复
-
Windows: CUDA Backend Fails to Initialize RTX 4050 While Vulkan Backend Works可能已有人在做 @pskrunner14 于 4 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 38/100
NVIDIA/NeMo-Speech.cpp#96 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 55/100
NVIDIA/NeMo-Speech.cpp#94 ·
维护者通常 1 天内回复
-
难度 5/5 一周以上 新手友好度 25/100
NVIDIA/NeMo-Speech.cpp#93 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 57/100
NVIDIA/NeMo-Speech.cpp#92 ·
维护者通常 1 天内回复
查看 NVIDIA/NeMo-Speech.cpp 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 75/100
维护者通常 1 天内回复
-
[BUG] Segfault when all runs of a Complexity() benchmark are skipped可能已有人在做 @nitanshnagpal 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
维护者通常 1 天内回复
-
Code: Performance
难度 1/5 1 小时以内 新手友好度 88/100
CrimsonCrossBunker/Cataclysm-Cleanwater-Bomb#1167 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
meshcore-dev/MeshCore#3760 · 1 条评论 ·
维护者通常 1 天内回复