coveralls support

まず tox.ini の coveralls コマンドとハイフン付きのエントリを確認し、次に Taskcluster のログと Balrog の .taskcluster.yml の例を調べます。各スクリプトの README を確認して、古いバッジを特定します。coverage の送信が機能し、リポジトリ全体でのレポートとスクリプト単位でのレポートのどちらを採用するかについてプロジェクトとして明確な決定がなされていれば完了です。

Support for coveralls needs attention.

We generate coverage data for each script via tox and then call coveralls to submit coverage data to coverage.io, but that's all legacy code that doesn't quite work currently. It appears that this was originally set up when each script was in a separate github repo, and using TravisCI. Each script's README points to coveralls.io badges for the old repos (like mozilla-releng/balrogscript, rather than mozilla-releng/scriptworker-scripts/balrogscript); of course those are all dormant currently.

As far as I can tell, coveralls.io assumes a one-to-one correspondence between a coverage project and a github repo: We can monitor coverage for mozilla-releng/scriptworker-scripts, but not mozilla-releng/scriptworker-scripts/balrogscript on its own.

Currently each scriptworker-scripts 'coveralls' command fails with the error message:

coveralls.exception.CoverallsException: Not on TravisCI. You have to provide either repo_token in .coveralls.yml or set the COVERALLS_REPO_TOKEN env var.

This does not cause a job failure (because of the leading hyphens in the main tox.ini), but you can see the tox taskcluster logs for details. The repo token can be found on coveralls.io. The Balrog project provides an example of specifying COVERALLS_REPO_TOKEN in a taskcluster project:

https://github.com/mozilla-releng/balrog/blob/main/.taskcluster.yml

Open questions:

• Can we continue to call coveralls for each script, or do we need to call coveralls just once for scriptworker-scripts after each script has updated its coverage data?

• Is there a way to track coverage per script, or do we just get one badge and report per repo? Can we live with that?