SonarCloud reports 0% coverage despite tests being generated and run

Inizia con .sonarcloud.properties e .github/workflows/pr.yaml, quindi esamina i comandi di test esistenti che producono coverage/lcov.info. Conferma come il workflow esegue attualmente i test e come uno step dello scanner SonarCloud può consumare quel report. Il lavoro è completato quando il percorso del report è configurato, è presente un’analisi basata sulla CI o il consumo dell’artefatto e un PR con modifiche testate mostra una copertura SonarCloud diversa da zero.

What happened

PR #202 added 205 lines of tests in src/commands/new/command.test.ts covering all three plugin types, validation, error cleanup, and interactive prompting. The CI checks job runs yarn backstage-cli repo test --coverage and the tests job runs yarn test (which expands to backstage-cli package test --coverage), so coverage data IS generated. However, SonarCloud's quality gate comment reported "0.0% Coverage on New Code" — and the quality gate still passed, providing false confidence about test coverage.

Investigation found three gaps:

1. .sonarcloud.properties does not set sonar.javascript.lcov.reportPaths — SonarCloud has no idea where to find coverage files.
2. No sonarsource/sonarcloud-github-action step exists in any CI workflow — SonarCloud is running in Automatic Analysis mode, which performs its own checkout and cannot access CI-generated coverage artifacts.
3. Coverage output from backstage-cli package test --coverage (written to coverage/lcov.info) is never uploaded as a CI artifact.

This affects all PRs, not just #202 — coverage has likely been invisible to SonarCloud for the entire project history.

What could go better

Coverage visibility is a foundational quality signal. Without it, SonarCloud's quality gate is checking only duplication and static analysis — its coverage assessment is meaningless. This gap means:

• Neither the review agent nor humans can cross-reference SonarCloud coverage data to assess test adequacy.
• The quality gate provides false assurance ("passed" despite 0% coverage).
• Coverage trends over time are not tracked, making regressions invisible.

Confidence: High. The .sonarcloud.properties file was directly inspected and contains only a CPD exclusion. The CI workflows were read and confirmed to have no SonarCloud scanner step. The --coverage flag in test scripts confirms coverage data is generated but never consumed.

This is a systemic repo-level issue, not specific to PR #202. It likely predates the fullsend integration.

Proposed change

Two changes needed in redhat-developer/rhdh-cli:

1. Add coverage report path to .sonarcloud.properties:

   sonar.javascript.lcov.reportPaths=coverage/lcov.info
   

2. Switch from Automatic Analysis to CI-based analysis by adding a SonarCloud scanner step to .github/workflows/pr.yaml after the test step (or uploading coverage as an artifact and running the scanner in a dependent job). Automatic Analysis cannot access CI-generated coverage files — CI-based analysis is required.

Alternatively, if staying with Automatic Analysis is preferred, the project should document that SonarCloud coverage data is not available and consider whether the quality gate should enforce a coverage threshold once CI-based analysis is enabled.

Validation criteria

After the fix: (1) SonarCloud reports non-zero coverage on PRs that add or modify tested code. (2) The sonar.javascript.lcov.reportPaths property is set in .sonarcloud.properties. (3) A SonarCloud scanner step exists in CI, or coverage is uploaded as an artifact consumed by the scanner. Verify on the next 2-3 PRs that include test changes.

Generated by retro agent from https://github.com/redhat-developer/rhdh-cli/pull/202