Reduce Solr JVM heap to leave room for OS page cache
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 15/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 停滞
- 技术栈
- helm, python
- 领域
- infrastructure, search
调研方向
The issue says the server-side heap setting lives in the Helm chart, which is outside this repository's scope; it names no chart files or tests to inspect here. Confirm the deployment chart and its memory settings with the maintainers, then verify after deployment using /status?full=true and the listed heap, free-memory, and slow-query metrics.
由索引模型根据 Issue 内容生成。
描述
Summary
Production /status is reporting a JVM heap that consumes nearly all of the host RAM, leaving the OS page cache too small to hold meaningful portions of the index. This is the most likely root cause of the bimodal latency we are seeing on the v1.5.2 instance:
- p50 = 78 ms (warm path)
- p95 = 4.3 s
- p99 = 42 s
- 12 % of queries classified as "very slow"
Evidence (from /status?full=true)
| Metric | Value |
|---|---|
solr.os.total_physical_memory_bytes |
34,359,738,368 (≈ 32 GiB) |
solr.jvm.heap_max_bytes |
32,212,254,720 (≈ 30 GiB) |
solr.os.free_physical_memory_bytes |
647,168 (≈ 632 KB) |
solr.os.physical_memory_used_pct |
1.0 (100 %) |
solr.size |
142.17 GB |
solr.os.process_cpu_load / system_cpu_load |
0 |
The heap is consuming ~94 % of host RAM. Lucene relies on the OS page cache for index segment access — with a 142 GB index against ~2 GiB of free RAM for filesystem cache, query tail latencies are dominated by disk paging. CPU load reading 0 while latency is high is consistent with I/O wait.
Suggested action (Helm chart)
- Drop the heap (
-Xmx) to ~12–16 GiB. - Increase node memory request/limit so at least 64 GiB (ideally more) is available, leaving ≥ 48 GiB for the OS page cache.
Solr/Lucene operations guidance recommends a small heap and a large filesystem cache for read-heavy workloads. The data-loading/Dockerfile currently sets -m 64G and data-loading/Makefile sets SOLR_MEM=220G — those are loader-side; the server-side setting lives in the Helm chart and is out of scope for this repo.
How to verify after deploy
After the change, hit /status?full=true and confirm:
solr.jvm.heap_max_bytesmatches the new-Xmxsolr.os.free_physical_memory_bytesis in the tens of GiB, not hundreds of KiBrecent_queries.latency_buckets.very_slow_pctdrops from 0.12 toward something close to 0
- 主要语言
- Python
- 星标
- 5
- 派生
- 4
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 没有贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
NCATSTranslator/NameResolution 的其他 Issue
-
难度 5/5 一周以上 新手友好度 35/100
-
难度 5/5 一周以上 新手友好度 35/100
-
难度 4/5 3-5 天 新手友好度 43/100
-
难度 3/5 1-2 天 新手友好度 66/100
-
难度 5/5 一周以上 新手友好度 25/100
查看 NCATSTranslator/NameResolution 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 86/100
benchflow-ai/skillsbench#1090 ·
-
Missing unit tests for human-readable date/time formatters in utils/datetime.py可能已有人在做 @prabhleen2317 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
apache/iceberg-python#4387 ·
维护者通常 1 天内回复
-
[BUG] Hermes MCP adapter drops cwd, extra passthrough keys and the SSE transport when writing config.yaml可能已有人在做 @Parth-Vasave 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复
-
backend bug
难度 2/5 1-3 小时 新手友好度 68/100
-
难度 2/5 1-3 小时 新手友好度 78/100
NVIDIA/numba-cuda-mlir#467 · 1 条评论 ·
维护者通常 3 天内回复