Subtests to skip with offline compiler

Open
#2,844 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp

Research direction

Start by reading testHarness.h, the check_compiler_available macro, and the test registration path through test_definition and callSingleTestFunction. Compare the proposed runtime check with registration metadata, then review how offline compilation mode currently uses subtests_to_skip_with_offline_compiler. Done means online compiler requirements are checked per test without subtest name collisions; the payload also proposes an OpenCL layer CI check.

Written by the indexing model from the issue text.

Description

The list subtests_to_skip_with_offline_compiler is very error prone. It is easy to forget to add a new test to that list, and as the list uses subtest names, we can easily have a subtest skipped because it has the same name in 2 different suites (ex: "get_program_info_kernel_names" from test_api & test_compiler).

Ideas:

1. Check gCompilationMode directly inside check_compiler_available

Tests that bypass create_single_kernel_helper and call clCreateProgramWithSource directly already need check_compiler_available(device) (recently moved to testHarness.h). Extending this macro (or replacing it with REQUIRE_ONLINE_COMPILER(device)) to also return TEST_SKIPPED_ITSELF when gCompilationMode != kOnline removes the need for a separate subtests_to_skip_with_offline_compiler list and avoids cross-suite name collisions.

2. Declare online-compiler requirements via REGISTER_TEST metadata

Extend test_definition with a flag indicating that a test requires online compilation, exposed via a dedicated registration macro (e.g., REGISTER_ONLINE_COMPILER_TEST). callSingleTestFunction can then automatically check both CL_DEVICE_COMPILER_AVAILABLE and gCompilationMode == kOnline on the specific test object, keeping the requirement co-located with the test definition and avoiding global subtest name lookups.


Testing:

Use an OpenCL layer to reject online compilation in CI

We could use an OpenCL layer when running in offline compilation mode to report CL_DEVICE_COMPILER_AVAILABLE = CL_FALSE and force online compilation calls (clCreateProgramWithSource, clBuildProgram from source, clCompileProgram) to fail with CL_COMPILER_NOT_AVAILABLE. Enabling this layer in a CI job running in offline compilation mode would immediately fail any new test that uses online compilation without properly skipping itself.

Dominant language
C++
Stars
234
Forks
240
Avg merge
6d 12h
Merged PRs (30d)
15

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from KhronosGroup/OpenCL-CTS

All issues in KhronosGroup/OpenCL-CTS

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.