[Question][framework] Configuring DevLake for ~6.5k GitLab repos and company-wide metrics
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 25/100
- Issue 类型
- 文档
- 描述清晰度
- 需要澄清
- 活跃度
- 活跃
- 技术栈
- gitlab, mysql
调研方向
该 issue 未指定源文件、测试或入口点。首先查看相关 issue #8448、#8802 和 #8260,然后收集有关大型 GitLab 部署、项目和 blueprint 布局、流水线并行性以及共享数据库限制的证据。完成的标准是记录受支持的规模、配置指南、瓶颈,以及是否支持多实例共享 MySQL 的运行方式。
由索引模型根据 Issue 内容生成。
描述
We are sizing a DevLake deployment for a large GitLab estate and want to know whether a single instance is expected to handle this, and how we should configure projects/blueprints if so.
Scale
- One product org already has ~6,500 GitLab repos; we need to plan for tens of thousands across the company.
- Teams are disjoint (no shared repos between team projects).
- We still need org- and company-wide metrics (e.g. Cycle Time) from one database. We do not use Grafana; a metrics API reads the same MySQL.
- Splitting into isolated DevLake+MySQL stacks would speed collection, but then we could not compute org- or company-wide Cycle Time with a single query against one database.
What we think is the intended setup (please correct us) :
- One lake process, one MySQL (we know a second instance on the same DB_URL hits the exclusive _devlake_locking_stub lock).
- Many team-sized projects/blueprints (tens to ~150 repos each), not one project with 6,500 scopes.
- PIPELINE_MAX_PARALLEL > 1, staggered crons, incremental sync, skip heavy gitextractor options if needed.
Questions
- Has anyone run DevLake successfully at a few thousand GitLab repos on one instance? What project size, PIPELINE_MAX_PARALLEL, and sync policy actually worked?
- Is the guidance above right, or is there a better project/blueprint layout for this?
- At this scale, is the bottleneck expected to be the single runner (sequential blueprints / sequential GitLab stages) rather than MySQL?
- If one instance cannot keep a daily incremental cycle, is the intended path still “more hardware on one process”, or is multi-instance sharing one DB something the project would consider?
Related: #8448, #8802, #8260
- 主要语言
- Go
- 星标
- 3.1k
- 派生
- 812
- 平均合并
- 1 天 23 小时
- 30 天内合并 PR
- 51
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/devlake 的其他 Issue
-
type/bug
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 4/5 3-5 天 新手友好度 45/100
-
难度 5/5 一周以上 新手友好度 35/100
-
type/bug
难度 4/5 3-5 天 新手友好度 52/100
-
难度 5/5 一周以上 新手友好度 35/100
相似的 Issue
-
bug github_actions
难度 2/5 1-3 小时 新手友好度 75/100
registrystack/registry-stack#1393 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
JakeChampion/lang#10213 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
oasisprotocol/oasis-sdk#2523 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 2/5 1-3 小时 新手友好度 70/100