Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

[Feature]: component test: discover-only JSON output and SSH mode

Đang mở
  • Có gì không ổn?
#374 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 6 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
15/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
go
Lĩnh vực
cli, testing-qa

Hướng nghiên cứu

This is design work, so start by reading docs/user/explanation/tmt-tests.md, especially the S1–S9 cases on where test files come from, and then the existing azldev component test implementation. Settle the open question about the source for components without a catalog entry, and agree on the --discover-only -O json schema and the --provision connect flags with maintainers. Done means a discussion decision, not a patch.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement

Is your feature request related to a problem? Please describe.

Our internal test generation tooling (azurelinux-test) builds behavior and fmt tests for
Azure Linux 4.0 packages, then generates and runs new tests. It has its own tmt code in two places, which duplicates
what azldev component test now supports:

  • Discovery. It clones a package's Fedora and CentOS Stream repositories, runs
    tmt run --until discover discover, and maps every discovered test back to the repository and commit it came from.
    azldev component test cannot report discovered tests without running them.
  • Execution. It runs generated tests through tmt on a guest provisioned separately and reached over SSH.
    azldev component test supports only virtual, which needs QEMU and /dev/kvm, and local, which needs an
    Azure Linux 4 host and changes that host. Neither works from Windows, macOS, a KVM-less CI runner, or a container.
    Running in a container is also unfaithful for tests that rely on systemd or the kernel.

I would like to drop our implementations and call azldev instead (already a runtime dependency of the test gen code). I'm happy to implement this, but want to make sure it will fit well.

Describe the solution you'd like

  1. Discover-only mode with JSON output. Resolve a component's plans exactly as a run would, stop after
    discover, and print each selected test pinned to the repository and commit its files actually came from. That is
    often not the plan's repository; the S1–S9 cases in docs/user/explanation/tmt-tests.md are what make it hard.
  2. --provision connect. Pass through tmt's own SSH provisioner (provision --how connect) to target any
    reachable Azure Linux 4 guest. tmt runs on the controller, so the guest needs sshd and rsync (which tmt
    installs if it is missing) plus whatever the test uses, but not tmt itself.
  3. Optional --rpm. When omitted (or maybe with --host-rpms-only as a mutually exclusive flag?), uses the test environment's currently installed packages.
  4. Per-test results with -O json. Report name, result, exit code, duration, and log path.

If it involves a CLI change, suggest it here:

# 1. list the tests a run would execute, without provisioning anything
azldev component test openssl --discover-only -O json

# 2-4. run a generated tree on an existing guest, against its installed packages
azldev component test openssh --provision connect --guest 10.0.0.5 --user azureuser --key ~/.ssh/id_ed25519 \
  --source-dir ./generated-tests --plan /plans/generated -O json

Proposed discovery output:

{
  "component": "openssl",
  "plans": [{ "name": "/plans/ci", "source": { "git-url": "...", "ref": "<sha>" } }],
  "tests": [
    {
      "name": "/Sanity/basic",
      "plans": ["/plans/ci", "/plans/tier1"],
      "git-url": "https://gitlab.com/redhat/centos-stream/tests/openssl.git",
      "ref": "<sha or null>",
      "path": "Sanity/basic",
      "sources": ["Sanity/basic/main.fmf"],
      "test": "./runtest.sh",
      "require": ["openssl"],
      "duration": "5m"
    }
  ],
  "unreachable": [{ "git-url": "https://gitlab.com/redhat/rhel/tests/openssl", "reason": "clone failed" }]
}

Describe alternatives you've considered

  • Keep our own tmt code. It works, but it duplicates azldev's pinning and will drift from the tests azldev
    actually runs. We also guess the Fedora/CentOS locations and use the branch head, where azldev knows each component's
    upstream and locked commit.
  • QEMU in a container. Requires /dev/kvm passthrough and nested virtualization; without them software emulation
    is far too slow. It still excludes most Windows and macOS setups.
  • --provision local in a container. The container is not an Azure Linux 4 system and has no systemd as PID 1,
    so tests fail or skip for reasons unrelated to the package.
  • A hand-rolled SSH harness. This is what we do now: copy the tree to the guest and run tmt there with local
    provisioning. It requires tmt and Python on the guest and reimplements result collection.

Use case / impact

  • Gather and coverage mapping in azurelinux-test get the same test set azldev runs, pinned to the packaged
    version. Once Azure Linux fills in its tmt catalog, they also get the plans Azure Linux itself declares.
  • Generated-test execution can target any Azure Linux 4 guest: Hyper-V, Azure, or bare metal. That covers Windows
    and macOS developers and CI without KVM, and suits the headless container-host image because the guest needs little.
  • azldev users get a way to preview and audit what a plan selects, and where each test comes from, before
    provisioning anything.

azldev version

v1.1.0 (c7c34596, the commit pinned by the Azure Linux 4.0 branch's .azldev-version).

Additional context

Open question: which source should discover-only use for a component with no catalog entry? We would suggest the
component's upstream dist-git at its locked commit, with additional locations such as the CentOS Stream tests
repository as an opt-in.

Ngôn ngữ chính
Go
Star
18
Fork
29
Merge trung bình
5 ngày 8 giờ
Pull request đã merge (30 ngày)
18

Chuẩn bị môi trường

Mở trong Codespaces

Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của microsoft/azure-linux-dev-tools

Tất cả issue của microsoft/azure-linux-dev-tools

Issue tương tự

Thêm issue về Go

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.