Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Feature]: component test: discover-only JSON output and SSH mode

Abierto
  • ¿Algo no está bien?
#374 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 7 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
go
Área
cli, testing-qa

Línea de trabajo

This is design work, so start by reading docs/user/explanation/tmt-tests.md, especially the S1–S9 cases on where test files come from, and then the existing azldev component test implementation. Settle the open question about the source for components without a catalog entry, and agree on the --discover-only -O json schema and the --provision connect flags with maintainers. Done means a discussion decision, not a patch.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Is your feature request related to a problem? Please describe.

Our internal test generation tooling (azurelinux-test) builds behavior and fmt tests for
Azure Linux 4.0 packages, then generates and runs new tests. It has its own tmt code in two places, which duplicates
what azldev component test now supports:

  • Discovery. It clones a package's Fedora and CentOS Stream repositories, runs
    tmt run --until discover discover, and maps every discovered test back to the repository and commit it came from.
    azldev component test cannot report discovered tests without running them.
  • Execution. It runs generated tests through tmt on a guest provisioned separately and reached over SSH.
    azldev component test supports only virtual, which needs QEMU and /dev/kvm, and local, which needs an
    Azure Linux 4 host and changes that host. Neither works from Windows, macOS, a KVM-less CI runner, or a container.
    Running in a container is also unfaithful for tests that rely on systemd or the kernel.

I would like to drop our implementations and call azldev instead (already a runtime dependency of the test gen code). I'm happy to implement this, but want to make sure it will fit well.

Describe the solution you'd like

  1. Discover-only mode with JSON output. Resolve a component's plans exactly as a run would, stop after
    discover, and print each selected test pinned to the repository and commit its files actually came from. That is
    often not the plan's repository; the S1–S9 cases in docs/user/explanation/tmt-tests.md are what make it hard.
  2. --provision connect. Pass through tmt's own SSH provisioner (provision --how connect) to target any
    reachable Azure Linux 4 guest. tmt runs on the controller, so the guest needs sshd and rsync (which tmt
    installs if it is missing) plus whatever the test uses, but not tmt itself.
  3. Optional --rpm. When omitted (or maybe with --host-rpms-only as a mutually exclusive flag?), uses the test environment's currently installed packages.
  4. Per-test results with -O json. Report name, result, exit code, duration, and log path.

If it involves a CLI change, suggest it here:

# 1. list the tests a run would execute, without provisioning anything
azldev component test openssl --discover-only -O json

# 2-4. run a generated tree on an existing guest, against its installed packages
azldev component test openssh --provision connect --guest 10.0.0.5 --user azureuser --key ~/.ssh/id_ed25519 \
  --source-dir ./generated-tests --plan /plans/generated -O json

Proposed discovery output:

{
  "component": "openssl",
  "plans": [{ "name": "/plans/ci", "source": { "git-url": "...", "ref": "<sha>" } }],
  "tests": [
    {
      "name": "/Sanity/basic",
      "plans": ["/plans/ci", "/plans/tier1"],
      "git-url": "https://gitlab.com/redhat/centos-stream/tests/openssl.git",
      "ref": "<sha or null>",
      "path": "Sanity/basic",
      "sources": ["Sanity/basic/main.fmf"],
      "test": "./runtest.sh",
      "require": ["openssl"],
      "duration": "5m"
    }
  ],
  "unreachable": [{ "git-url": "https://gitlab.com/redhat/rhel/tests/openssl", "reason": "clone failed" }]
}

Describe alternatives you've considered

  • Keep our own tmt code. It works, but it duplicates azldev's pinning and will drift from the tests azldev
    actually runs. We also guess the Fedora/CentOS locations and use the branch head, where azldev knows each component's
    upstream and locked commit.
  • QEMU in a container. Requires /dev/kvm passthrough and nested virtualization; without them software emulation
    is far too slow. It still excludes most Windows and macOS setups.
  • --provision local in a container. The container is not an Azure Linux 4 system and has no systemd as PID 1,
    so tests fail or skip for reasons unrelated to the package.
  • A hand-rolled SSH harness. This is what we do now: copy the tree to the guest and run tmt there with local
    provisioning. It requires tmt and Python on the guest and reimplements result collection.

Use case / impact

  • Gather and coverage mapping in azurelinux-test get the same test set azldev runs, pinned to the packaged
    version. Once Azure Linux fills in its tmt catalog, they also get the plans Azure Linux itself declares.
  • Generated-test execution can target any Azure Linux 4 guest: Hyper-V, Azure, or bare metal. That covers Windows
    and macOS developers and CI without KVM, and suits the headless container-host image because the guest needs little.
  • azldev users get a way to preview and audit what a plan selects, and where each test comes from, before
    provisioning anything.

azldev version

v1.1.0 (c7c34596, the commit pinned by the Azure Linux 4.0 branch's .azldev-version).

Additional context

Open question: which source should discover-only use for a component with no catalog entry? We would suggest the
component's upstream dist-git at its locked commit, with additional locations such as the CentOS Stream tests
repository as an opt-in.

Lenguaje dominante
Go
Estrellas
18
Forks
29
Merge medio
6 d 19 h
PR fusionados (30 d)
19

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de microsoft/azure-linux-dev-tools

Todos los issues de microsoft/azure-linux-dev-tools

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.