Periodically find and queue missing compiler compatibility checks

未关闭
  • 哪里不对？
#804 9 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
22/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
领域
backend

调研方向

This is design work, so start by reading the linked #800, #801 and #803 to see what those changes already cover. The issue names no files: look for where the publish job queues compiler matrix jobs and where the existing job queue is written. Done means a design decision on the stateless sweep, the queue-insertion rule, and whether scheduling moves outside the publish job's success boundary, agreed with maintainers before any code is written.

由索引模型根据 Issue 内容生成。

描述

A package can finish publishing, but some of its compiler compatibility checks may never get queued. For example, the server might time out or restart while finding the checks to run. The package is available to users, but its list of supported compilers can remain incomplete indefinitely.

This happened with [email protected] in #800.

#801 makes follow-up scheduling faster and inserts jobs one at a time. That saves jobs already queued before an interruption, but it cannot recover checks it never reached. #803 improves solver performance, but faster solving alone does not repair missing checks.

So the basic problem remains: how do we detect and fix the unavoidable case where a compiler matrix job might not get queued for a published package? And how do we do it without introducing new state?

I think we can accomplish this with a regular, stateless sweep over the registry. The basic idea is to run a small repair command on a regular interval, for example through a systemd timer. Each run starts fresh and asks: Which published package versions still need compiler checks?

For each published package version and each supported compiler:

  1. If the package metadata already lists that exact compiler, skip it.
  2. If a matrix job already exists for that package version and compiler, skip it.
  3. Otherwise, try to find dependency versions that allow a build with that compiler.
  4. If a build plan is found, immediately insert a matrix job into the existing queue.
  5. If no plan is found, leave it alone and try again on a later sweep. A dependency may gain compiler support in the meantime.

The normal server executor still runs the compilation jobs and updates package metadata after successful compilation. The repair command does not compile packages or write compatibility metadata itself.

This survives interruptions itself. As a concrete example, suppose a sweep finds 100 missing checks and stops after queuing 40. The next sweep starts over, skips those 40 existing jobs, and finds the remaining checks. We do not need to remember where the previous sweep stopped.

This also finds old gaps, not just gaps caused by the latest publication or compiler release. It should examine all currently published versions, not only the newest version of each package.

The goal is eventual repair: as sweeps and compilation jobs complete, missing compatibility results are filled in wherever a build succeeds. A missing compiler entry is not proof that the package cannot compile with it, and a solved build plan is not proof of compatibility until compilation succeeds.

Side note: separating queueing from publication success

If publication finishes successfully, a later timeout while queuing compatibility checks currently makes the publish job fail. This is kind of misleading though; the compiler matrix is an eventually-correct best-effort thing, not a requirement for publishing. Compatibility scheduling should be outside the publication job's success/failure boundary and timeout. The periodic sweep can take responsibility for discovering that follow-up work.

The same principle applies to matrix jobs: once compilation succeeds and the compatibility metadata is successfully written/pushed, a later failure to schedule dependant checks should not turn that successful check into a failed job.

主要语言
PureScript
星标
95
派生
79
平均合并
16 小时 13 分钟
30 天内合并 PR
14

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

purescript/registry-dev 的其他 Issue

查看 purescript/registry-dev 的全部 Issue

相似的 Issue

更多 Backend & API Design Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。