/status/ runs a full SUM(size) over all artifacts on every call
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 领域
- backend, performance
调研方向
The full aggregate is in _disk_usage() in pulpcore/app/views/status.py, filtered by pulp_domain and run on every /status/ call when storage is not FileSystem. Read that function and how the storage type is checked, then pick one of the fixes in the issue (periodic task, cache, or an opt-out) after confirming the approach with maintainers. Done means /status/ no longer runs the full SUM(size) per request, with the storage figure still reported.
由索引模型根据 Issue 内容生成。
描述
Version
- pulpcore 3.95.2. The code is the same in 3.121.2 and on
main. - Deployed with pulp-operator, with object storage (Google Cloud Storage) and the default domain only.
Describe the bug
Each call to GET /pulp/api/v3/status/ runs an aggregate over the whole artifact table when the storage is not FileSystem. In pulpcore/app/views/status.py, _disk_usage() does this:
used = Artifact.objects.filter(pulp_domain=domain).aggregate(size=Sum("size", default=0))
This becomes:
SELECT COALESCE(SUM("core_artifact"."size"), $1) AS "size"
FROM "core_artifact"
WHERE "core_artifact"."pulp_domain_id" = $2::uuid
With one domain, the WHERE matches every row, so each call reads all artifacts. The cost grows with the number of artifacts.
The pulp-operator uses /status/ for both default API probes (liveness httpGet, readiness readyz.py). So the query runs on every probe of every API pod, not only when a user asks for the status. In our deployment, this load kept the database CPU high. During package uploads, /status/ became slower than the probe timeout, and the liveness probe restarted healthy API pods.
To Reproduce
- Use object storage (for example S3 or GCS) and a large
core_artifacttable. - Deploy with pulp-operator and the default probes, with many API replicas.
- Look at the query above in
pg_stat_statements, and at the database CPU.
Expected behavior
/status/ is cheap enough to call from health checks. The storage usage does not run a full aggregate on each call.
Some possible fixes:
- Compute the storage usage in a periodic task and store or cache the result.
/status/returns the stored value. - Cache the result of
_disk_usage()for some minutes. - Add a query parameter or a setting to leave
storageout of the response.
Additional context
- #7475 reports the same aggregate in the
otel_metricstask. Per the comments on #7475, the index from #7511 did not help. With one domain, an index-only scan still reads one entry for each artifact. - As a workaround, we override the probes in the
PulpCR: liveness uses/livez/, and readiness calls/status/less often. We opened an issue in pulp-operator to change the defaults: https://github.com/pulp/pulp-operator/issues/1696
- 主要语言
- Python
- 星标
- 626
- 派生
- 177
- 平均合并
- 16 小时 49 分钟
- 30 天内合并 PR
- 126
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
pulp/pulpcore 的其他 Issue
-
Content cache ignores the query string, so one path serves the response cached for another query未关闭
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
Issue
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
Feature
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
Issue Triage-Needed
难度 3/5 1-2 天 新手友好度 68/100
维护者通常 1 天内回复
-
Issue
难度 4/5 3-5 天 新手友好度 42/100
维护者通常 1 天内回复
相似的 Issue
-
Import errors are swallowed: PerthImplicitWatermarker silently becomes None可能已有人在做 @JonRC 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
resemble-ai/Perth#23 ·
-
correction metadata
难度 2/5 1 小时以内 新手友好度 65/100
acl-org/acl-anthology#10753 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
earthtojake/text-to-cad#758 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 70/100
DataTalksClub/faq#545 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复