altool runtests hangs after TestRunnerHub Initialize fails with application authentication
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- powershell
- Domain
- authentication, cli, testing
Research direction
Start with the altool.dll runtests entry point and the TestRunnerHub Initialize and cleanup flow described in the report. Compare application-token and delegated authentication, then verify that an initialization failure is reported with a nonzero exit instead of leaving the process running; a minimal public AL test extension is still needed to confirm the reproduction.
Written by the indexing model from the issue text.
Description
1. Describe the bug
When running altool runtests against a Business Central Online sandbox using an application access token (client credentials), the tool connects to TestRunnerHub but does not complete or return a useful error.
An isolated diagnostic call using the installed Microsoft AL libraries exposed a server exception from Initialize. Disposing the failed runner also did not complete within a five-second diagnostic deadline.
Using delegated user authentication with the same environment, company, tool version and installed extensions, all ten test methods pass.
We have not confirmed whether application authentication is supported for this endpoint. Regardless, the CLI should report the initialization failure and exit instead of hanging.
2. To Reproduce
- Use a Business Central Online sandbox with a published AL test extension.
- Configure an enabled Entra application in BC with:
- Application API permissions:
API.ReadWrite.AllandAutomation.ReadWrite.All, with tenant consent. - BC permission sets:
D365 AUTOMATIONandEXTEN. MGT. - ADMIN, without a company restriction.
- Application API permissions:
- Obtain a client-credentials access token for the target tenant and the Business Central resource.
- Supply it to the CLI through the
BC_ACCESS_TOKENenvironment variable. - Run:
dotnet "<AL-extension-directory>/bin/altool.dll" runtests `
--testgroups "<path-to-test-groups.json>" `
--tenant "<tenant-id>" `
--environmentname "<sandbox-name>" `
--environmenttype Sandbox `
--company "<company-name>"
- Observe that the process connects but does not finish. In our pipeline, we cancelled it after more than five minutes.
- Repeat using normal delegated AL authentication against the same target. The tests complete successfully.
To isolate the failure, we also used the installed Microsoft AL libraries to connect and invoke only:
Initialize("<company-name>", "", CoverageMode.None)
This reproduced the exception before any test method was invoked.
The original tests belong to a private customer extension. We have not yet reproduced this with a separate minimal public AL test extension. The initialization-only diagnostic isolates the failure from execution of the test methods.
3. Expected behavior
If application authentication is supported, initialization and test execution should complete with the required permissions.
If it is unsupported, or initialization otherwise fails, the CLI should report a clear error and exit with a nonzero exit code. Cleanup should not prevent the error from being reported.
4. Actual behavior
With application authentication:
- Extension publication through the Automation API succeeds.
- Developer metadata retrieval and the TestRunnerHub connection succeed.
- The isolated Initialize call returns:
Microsoft.AspNetCore.SignalR.HubException:
An unexpected error occurred invoking 'Initialize' on the server.
- Disposing the failed runner exceeds a five-second diagnostic deadline.
- The normal CLI remains running without producing a completed test result.
With delegated authentication:
- Initialization succeeds.
- All ten named test methods pass.
We verified the application's BC permission assignments through the Automation API. D365 AUTOMATION includes execute permission on Codeunit ID 0 (all codeunits). However, we have not established equivalent effective permissions or entitlements between the application identity and the delegated administrator.
The Admin Center telemetry query covering the failure returned HTTP 200 with no entries. Application Insights is not configured, so the underlying server exception remains unknown.
An external process timeout now prevents our pipeline from hanging indefinitely, but it does not resolve the initialization failure.
5. Versions:
- AL Language: 18.0.2732683
- Visual Studio Code: Not involved in the failing reproduction;
altool.dllwas invoked directly usingdotnet. - Business Central: Online sandbox, application version 28.5.54151.54677
- .NET runtime: 10.0.12
- List of Visual Studio Code extensions that you have installed: The reproduction runs outside the VS Code extension host. A separate IDE reproduction with all other extensions disabled has not been performed.
- Operating System:
- Windows — build 26200
- Linux
- MacOS
Final Checklist
- Search the issue repository to ensure you are reporting a new issue
- Reproduce the issue after disabling all extensions except the AL Language extension
- Simplify your code around the issue to better isolate the problem
Isolation performed: reproduced the Initialize failure without invoking any test methods. A minimal public AL test project is not yet available.
- Dominant language
- PowerShell
- Stars
- 884
- Forks
- 285
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/AL
-
accepted al-tools bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 5/5 Over a week Newbie friendliness 8/100
-
accepted
Difficulty 5/5 Over a week Newbie friendliness 12/100
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Similar issues
-
area/frontend area/v2 kind/bug priority/needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
kubeflow/notebooks#1498 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
splunk/security_content#4334 ·
Maintainers usually reply within 1 day
-
Streamable HTTP client: a 401 or 403 with a JSON-RPC error body and no WWW-Authenticate loses its HTTP statusPossibly taken A pull request linked to this issue is open or already merged. Openbug P2 ready for work T-security T-transport
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
modelcontextprotocol/rust-sdk#1339 ·
Maintainers usually reply within 3 days
-
bug : find_key() compares kty against "ocy" instead of "oct", breaking kid-less HS256 verificationOpen
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
OpenPrinting/cups#1756 ·
Maintainers usually reply within 1 day