[Feature]: Add CI checks for pull requests

Open
#52 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
github-actions, json, python, yaml
Domain
ci-cd, tooling

Research direction

Read .github/workflows/update_plugin.yaml and script/update_plugin.py first to see how sources are validated and how plugins/index.json is rebuilt, then compare a few files under sources/ with their plugins/*.json counterparts. The issue's own checklist (steps 1 and 2) is the scope to agree on before coding, since the test block and smoke test are still open questions for the maintainers. Done looks like a new pull-request workflow that fails on invalid or orphaned source files.

Written by the indexing model from the issue text.

Description

enhancement help wanted
1. Your usage scenarios?

Pull requests in this repo get no automated checks. I checked PRs 30, 32, 34, 41 and 42, and none of them has a status or a check run. PR 42 was approved and merged with no checks at all. PR 34, a Dependabot bump of actions/checkout, is still open with no checks.

The only workflow is .github/workflows/update_plugin.yaml. It runs every hour and on manual dispatch, never on pull requests. It reads sources/, updates plugins/*.json, rebuilds plugins/index.json, commits the result to the branch and deploys Pages. The registry docs say a merged source is published automatically and checked every hour, so the generated files reach the site without a review.

Some problems I found while reading the repo:

  • PR 35 changed plugins/chaosblade.json and plugins/index.json directly, with no file under sources/. The hourly job rebuilds the index from sources/, so chaosblade is missing from plugins/index.json now, and the job never updates its file.
  • The registry docs describe a test block in the source file format (version, check, resultRegx). Nothing in this repo reads it.
  • As far as I can read script/update_plugin.py, a source whose manifest or zip download fails is only printed. The run still exits 0, and that plugin is left out of plugins/index.json for that run.
  • ad-m/github-push-action@master is not pinned to a release or a commit.
2. What is your expected outcome?

A workflow that runs on pull requests and has to pass before merge. These are the checks I would start with, in this order:

  1. For each changed sources/*.json: valid JSON, name and manifestUrl present, and the file name equal to name.
  2. Every plugins/*.json has a source file, and every source has an entry in plugins/index.json.
  3. Run script/update_plugin.py in the check, and fail if any source fails.
  4. For each changed source, install test.version with vfox, run test.check and match test.resultRegx. This needs a vfox binary and a real download, so it is slower and can be flaky. I would start with Linux only.
  5. Pin third-party actions to a full commit SHA.

Questions for the maintainers:

  • Should the hourly job fail when a source fails, or only report it?
  • Should the test block run in CI, and should new sources be required to have one?
  • Which systems should the smoke test cover?

Once we agree on the scope, I can send a PR for steps 1 and 2.

Dominant language
Python
Stars
32
Forks
21
Avg merge
4h 55m
Merged PRs (30d)
3

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from version-fox/vfox-plugins

All issues in version-fox/vfox-plugins

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.