Ansible/Kubernetes: helm-unittest suites in charts/*/unittests via ct.yaml additional-commands, yamllint as an Ansible linter, molecule's pytest verifier, antsibull-nox

Start with project_validator/ecosystems/kubernetes.py around lines 116-118, where the helm-unittest pattern is defined, and with the docs/ecosystems/ansible.md note on yamllint. The issue asks for several separate changes: reading chart-testing's ct.yaml additional-commands, widening the unittests/ pattern, crediting shellcheck and yamllint, and recognising antsibull-nox sessions. Each needs a design decision first, so confirm the scope with a maintainer. Done means the corpus cases for these repositories produce the expected statuses.

prometheus-community/helm-charts runs 118 helm-unittest suites (charts/*/unittests/**/*.yaml) and shellcheck through chart-testing's ct.yaml additional-commands (helm unittest --strict …); the config file is never read and the pattern requires tests/*_test.yaml. yamllint is the one linter every Ansible role in the corpus runs (yamllint ., pre-commit hook, run_yamllint = true) but is not an Ansible static-analysis claim. kubespray's one Python "test" is a molecule/testinfra verifier that molecule test runs. community.general's entire CI is antsibull-nox (declared in the repo's own config), the standard driver of ~100 ansible-collections repos.

Evidence from the corpus audit

ansible-collections/community.general / tests-run, static-analysis, type-checking, code-formatting

• kind: UNSUPPORTED
• actual: ERROR (strict) "not read: job ansible-test: it calls the reusable workflow
  ansible-community/antsibull-nox/.github/workflows/reusable-nox-matrix.yml@main"; with --allow-unknown:
  tests-run FAIL "Ansible: tests found in 281 file(s), but no CI configuration runs molecule test or ansible-test
  integration", static-analysis FAIL "Ansible: ansible-lint is configured (.ansible-lint) but does not run; Python:
  ruff check is configured (ruff.toml) but does not run", type-checking WARN "mypy is configured (.mypy.ini) but
  does not run", code-formatting FAIL "Python: ruff.toml is defined but CI does not run ruff format".
• expected: the reusable workflow is unreadable (see "Unreadable by nature"), but everything it runs is declared in
  the repository's own antsibull-nox configuration, the standard CI driver of the ansible-collections organisation
  (~100 collections). Recognising antsibull-nox.toml sessions ([sessions.ansible_lint],
  [sessions.ansible_test_sanity], [sessions.ansible_test_units], [sessions.ansible_test_integration],
  run_mypy = true, run_ruff_check = true, run_ruff_format = true, run_yamllint = true) when the workflow
  uses: ansible-community/antsibull-nox/... would answer all four checks.
• evidence: .github/workflows/nox.yml:28: uses: ansible-community/antsibull-nox/.github/workflows/reusable-nox-matrix.yml@main;
  antsibull-nox.toml: [sessions.lint] ... run_ruff_check = true / run_ruff_format = true / run_yamllint = true
  / run_mypy = true / mypy_config = ".mypy.ini"; [sessions.ansible_lint]; [sessions.ansible_test_sanity];
  [sessions.ansible_test_units] with [sessions.ansible_test_units.core_python_versions] "2.19" = ["3.8", "3.11", "3.13"];
  [sessions.ansible_test_integration]; noxfile.py present at the root.
• repro: galaxy.yml + antsibull-nox.toml with [sessions.ansible_test_units] + a workflow whose job uses: ansible-community/antsibull-nox/.github/workflows/reusable-nox-matrix.yml@main.
• confidence: medium (a gap, not a bug)

kubernetes-sigs/kubespray / tests-run (Python)

• kind: FALSE_POSITIVE
• actual: FAIL (component) "Python: the root: tests found in 1 file(s), but no CI configuration runs pytest, unittest
  or nox"
• expected: the one file is a molecule/testinfra verifier that molecule test runs (molecule's verify stage
  invokes pytest on molecule/<scenario>/tests/test_*.py); CI runs molecule (tests/scripts/molecule_run.sh from
  .gitlab-ci/molecule.yml), and the Ansible component of the same check is credited ("Ansible: tests found in 9
  file(s)"). Test files under molecule/*/tests/ belong to the molecule scenario, not to a pytest suite.
• evidence: find . -name 'test_*.py' -o -name '*_test.py' → only
  ./roles/bootstrap_os/molecule/default/tests/test_default.py; roles/bootstrap_os/molecule/default/molecule.yml
  present (scenario).
• repro: molecule/default/molecule.yml + molecule/default/tests/test_default.py + CI molecule test.
• confidence: low (molecule ≥ 6 made the testinfra verifier optional, so whether it runs depends on the scenario's
  verifier:; the scenario here declares none in the lines read)

prometheus-community/helm-charts / tests-run, static-analysis (Shell)

• kind: FALSE_NEGATIVE
• actual: tests-run "Kubernetes: tests found in 11 file(s)" (7 templates/tests/*.yaml hooks + the few
  tests/*_test.yaml); static-analysis FAIL "Shell: no static analyzer runs"; compiler-warnings FAIL "Shell: no
  shell lints found"; warnings-as-errors FAIL.
• expected: 118 helm-unittest suites under charts/*/unittests/**/*.yaml counted and credited as run, and shellcheck
  credited: chart-testing's configuration file runs helm unittest --strict and shellcheck on every changed chart
  through additional-commands, and ct lint --config .github/linters/ct.yaml runs in CI. The validator neither
  reads ct's config file (.github/linters/ct.yaml / ct.yaml, the only place additional-commands can live) nor
  recognises unittests/ as a helm-unittest directory (its pattern requires tests/*_test.yaml).
• evidence: .github/linters/ct.yaml:2-4: additional-commands: /
  - sh -ec "if [ -f '{{ .Path }}/ci/lint.sh' ]; then shellcheck '{{ .Path }}/ci/lint.sh'; bash '{{ .Path }}/ci/lint.sh'; fi" /
  - helm unittest --strict --file 'unittests/**/*.yaml' {{ .Path }}; .github/workflows/ci.yaml:76:
  run: ct lint --config .github/linters/ct.yaml; :63: run: helm plugin install https://github.com/helm-unittest/helm-unittest.git ...;
  find charts -path '*unittests/*_test.yaml' | wc -l → 118 (e.g. charts/prometheus-rabbitmq-exporter/unittests/service_test.yaml,
  charts/prometheus-stackdriver-exporter/unittests/deployment_namespace_test.yaml).
  Validator: project_validator/ecosystems/kubernetes.py:116-118: name="helm-unittest tests (tests/*_test.yaml)",
  path=re.compile(r"(?:^|/)tests/.+_test\.ya?ml$"); no reference to additional-commands or ct.yaml anywhere
  in project_validator/.
• repro: charts/a/Chart.yaml + charts/a/unittests/deployment_test.yaml (suite: ...) + ct.yaml with
  additional-commands: [helm unittest --strict --file 'unittests/**/*.yaml' {{ .Path }}] + a workflow
  run: ct lint --config ct.yaml.
• confidence: high

geerlingguy/ansible-role-docker, geerlingguy/ansible-for-devops, kubernetes-sigs/kubespray / static-analysis

• kind: UNSUPPORTED
• actual: ansible-role-docker static-analysis WARN "ansible-lint is configured (.ansible-lint) but does not run"
  (correct, see below) and data-wellformed PASS "22 YAML files parse" with no mention of yamllint;
  kubespray/pre-commit runs yamllint, ansible-static-lint (astl), misspell and a Jinja syntax check that are not
  credited anywhere.
• expected: not a wrong status, but yamllint is the one linter that every Ansible role in this batch runs
  (yamllint . in ansible-role-docker's CI, pip3 install ... yamllint in ansible-for-devops, a pre-commit hook in
  kubespray, run_yamllint = true in community.general) and the Data profile has nowhere to report it; the Ansible
  profile doc says "yamllint lints the YAML, which is the Data format's to ask for", but the Data format never asks.
• evidence: geerlingguy__ansible-role-docker/.github/workflows/ci.yml:32: run: pip3 install yamllint; :35-36:
  run: | / yamllint .; .yamllint:1: extends: default; kubernetes-sigs__kubespray/.pre-commit-config.yaml:17-20:
  - repo: https://github.com/adrienverge/yamllint.git ... - id: yamllint; :31-34:
  - repo: https://github.com/arhuman/ansible-static-lint ... - id: astl.
• repro: n/a (a gap)
• confidence: low (by design per docs/ecosystems/ansible.md; recorded because it is the ecosystem's universal tool)

Found by the regression-corpus audit (235 real repositories, one or more per ecosystem profile; corpus and expected results in corpus/). Evidence is quoted from the audited checkouts at the commits pinned in corpus/projects.toml; <checkouts>/<owner>__<repo> is such a checkout.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt