Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Apple][CoreCLR] Runtime-test merged runners skip every test: executable path is passed as args[0] and used as the test filter

Closed Beginner friendly
  • Is something wrong?
#134,766 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
csharp, ios, objective-c
Domain
mobile-dev, testing-qa

Research direction

Start with src/tasks/AppleAppBuilder/Templates/runtime-coreclr.m and inspect get_managed_args in util.m, then compare the CoreCLR argument handling with the Mono host and corerun. Verify the runtime-test lanes *_AllSubsets_CoreCLR_RuntimeTests no longer treat the executable path as a test filter and begin reporting executed tests instead of skipping every test.

Written by the indexing model from the issue text.

Description

area-Infrastructure-coreclr os-ios os-maccatalyst os-tvos untriaged
Description

On CoreCLR for iOS, iOS simulator, tvOS and maccatalyst, the runtime-test lanes (*_AllSubsets_CoreCLR_RuntimeTests in runtime-extra-platforms) report every test as skipped, with reason "No Known Skip Reason". No runtime tests actually run on these platforms.

Examples from build 1613653 (main, 2026-09-27):

Lane Helix job Regression_o_3 Regression_2
maccatalyst-arm64 0b9c9ad9-cb7f-448b-9b7b-b93a5523d429 0 pass / 167 skip 0 pass / 107 skip
iossimulator-arm64 37c0646d-11ad-4a07-a440-68280a0a1d99 0 pass / 167 skip 0 pass / 107 skip
tvos-arm64 aa23176f-37b8-42cc-946b-01f8805c8120 0 pass / 167 skip 0 pass / 107 skip

Other tvOS work items (Regression_1, Regressions, Methodical_d1, threading_group1, Interop) are also 0 passed and all skipped. The same holds in build 1598688 (2026-09-16). The app log shows [Runner executing: Run everything] followed by Tests run: 167 Passed: 0 ... Skipped: 167 within about 80 ms.

Cause (from code)
  1. The CoreCLR Apple host src/tasks/AppleAppBuilder/Templates/runtime-coreclr.m builds the managed arguments with get_managed_args (util.m). That copies all of [[NSProcessInfo processInfo] arguments], whose element 0 is the executable path.
  2. It passes that whole array to coreclr_execute_assembly, whose argv becomes managed Main's args. So args[0] is the executable path. The Mono host (runtime.m) passes the same array to mono_jit_exec, which treats argv[0] as the program name, so Mono is unaffected.
  3. The generated merged-runner Main for mobile calls XHarnessRunnerLibrary.RunnerEntryPoint.RunTests(RunTests, "<asm>", args.Length != 0 ? args[0] : null) (XUnitWrapperGenerator.cs). AppleEntryPoint treats that value as the only method to run: runner.SkipMethod(_methodNameToRun, isExcluded: false).
  4. No test's fully qualified name matches the executable path, so TestFilter.ShouldRunTest returns false for every test. Each one is reported skipped with an empty reason, which TestSummary prints as "No Known Skip Reason".

The library test lanes are not affected. AppleTestRunner only acts on testlib: and --verbose arguments.

Proposed fix

In runtime-coreclr.m, pass the managed arguments without the program name, matching corerun and the dotnet host:

coreclr_execute_assembly (coreclr_handle, coreclr_domainId, argi - 1, managed_argv + 1, path, &res);

Once fixed, these lanes will start running tests, and new failures are likely to show up.

[!NOTE]
This issue was generated with GitHub Copilot.

Dominant language
C#
Stars
18.3k
Forks
5.6k
Avg merge
2d 18h
Merged PRs (30d)
617

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/runtime

All issues in dotnet/runtime

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.