altool runtests hangs after TestRunnerHub Initialize fails with application authentication

Start with the altool.dll runtests entry point and the TestRunnerHub Initialize and cleanup flow described in the report. Compare application-token and delegated authentication, then verify that an initialization failure is reported with a nonzero exit instead of leaving the process running; a minimal public AL test extension is still needed to confirm the reproduction.

1. Describe the bug

When running altool runtests against a Business Central Online sandbox using an application access token (client credentials), the tool connects to TestRunnerHub but does not complete or return a useful error.

An isolated diagnostic call using the installed Microsoft AL libraries exposed a server exception from Initialize. Disposing the failed runner also did not complete within a five-second diagnostic deadline.

Using delegated user authentication with the same environment, company, tool version and installed extensions, all ten test methods pass.

We have not confirmed whether application authentication is supported for this endpoint. Regardless, the CLI should report the initialization failure and exit instead of hanging.

2. To Reproduce

1. Use a Business Central Online sandbox with a published AL test extension.
2. Configure an enabled Entra application in BC with:
   • Application API permissions: API.ReadWrite.All and Automation.ReadWrite.All, with tenant consent.
   • BC permission sets: D365 AUTOMATION and EXTEN. MGT. - ADMIN, without a company restriction.
3. Obtain a client-credentials access token for the target tenant and the Business Central resource.
4. Supply it to the CLI through the BC_ACCESS_TOKEN environment variable.
5. Run:

dotnet "<AL-extension-directory>/bin/altool.dll" runtests `
    --testgroups "<path-to-test-groups.json>" `
    --tenant "<tenant-id>" `
    --environmentname "<sandbox-name>" `
    --environmenttype Sandbox `
    --company "<company-name>"

6. Observe that the process connects but does not finish. In our pipeline, we cancelled it after more than five minutes.
7. Repeat using normal delegated AL authentication against the same target. The tests complete successfully.

To isolate the failure, we also used the installed Microsoft AL libraries to connect and invoke only:

Initialize("<company-name>", "", CoverageMode.None)

This reproduced the exception before any test method was invoked.

The original tests belong to a private customer extension. We have not yet reproduced this with a separate minimal public AL test extension. The initialization-only diagnostic isolates the failure from execution of the test methods.

3. Expected behavior

If application authentication is supported, initialization and test execution should complete with the required permissions.

If it is unsupported, or initialization otherwise fails, the CLI should report a clear error and exit with a nonzero exit code. Cleanup should not prevent the error from being reported.

4. Actual behavior

With application authentication:

• Extension publication through the Automation API succeeds.
• Developer metadata retrieval and the TestRunnerHub connection succeed.
• The isolated Initialize call returns:

Microsoft.AspNetCore.SignalR.HubException:
An unexpected error occurred invoking 'Initialize' on the server.

• Disposing the failed runner exceeds a five-second diagnostic deadline.
• The normal CLI remains running without producing a completed test result.

With delegated authentication:

• Initialization succeeds.
• All ten named test methods pass.

We verified the application's BC permission assignments through the Automation API. D365 AUTOMATION includes execute permission on Codeunit ID 0 (all codeunits). However, we have not established equivalent effective permissions or entitlements between the application identity and the delegated administrator.

The Admin Center telemetry query covering the failure returned HTTP 200 with no entries. Application Insights is not configured, so the underlying server exception remains unknown.

An external process timeout now prevents our pipeline from hanging indefinitely, but it does not resolve the initialization failure.

5. Versions:

• AL Language: 18.0.2732683
• Visual Studio Code: Not involved in the failing reproduction; altool.dll was invoked directly using dotnet.
• Business Central: Online sandbox, application version 28.5.54151.54677
• .NET runtime: 10.0.12
• List of Visual Studio Code extensions that you have installed: The reproduction runs outside the VS Code extension host. A separate IDE reproduction with all other extensions disabled has not been performed.
• Operating System:
  • Windows — build 26200
  • Linux
  • MacOS

Final Checklist

• Search the issue repository to ensure you are reporting a new issue
• Reproduce the issue after disabling all extensions except the AL Language extension
• Simplify your code around the issue to better isolate the problem

Isolation performed: reproduced the Initialize failure without invoking any test methods. A minimal public AL test project is not yet available.