Veeam plugin fails when Enterprise Manager and Veeam B&R are on separate hosts (Get-VBRJob over SSH not available)

Start with VeeamClient.java, especially getRepositoryNameFromJob and cloneVeeamJob, and the Veeam plugin configuration and documentation. Trace how the Enterprise Manager URL is also used as the SSH target, then check the plugin tests and supported Veeam version notes. Done means the separate-host setup can complete instance assignment, with a clear error when PowerShell is unavailable.

Summary

The Veeam B&R backup plugin uses the host from backup.plugin.veeam.url both for the Enterprise Manager REST API and for an SSH session in which it runs Veeam B&R PowerShell commands (e.g. Get-VBRJob). This implicitly requires Enterprise Manager and Veeam Backup & Replication (with its PowerShell module) to be installed on the same host.

A common Veeam architecture runs Enterprise Manager on a dedicated server that manages several backup servers. In this setup the REST part of the plugin works, but every operation that needs PowerShell fails, because the Veeam PowerShell module is not available on the Enterprise Manager host. As a result, instances cannot be assigned to a backup offering.

Environment

• Apache CloudStack 4.23.0.0, VMware vSphere (vCenter) zone
• Veeam Backup & Replication 13.1.1.18 (Windows Server), two backup servers on two sites
• Veeam Backup Enterprise Manager 13.1.1.18 on a separate Windows Server (central management node for both backup servers)
• OpenSSH Server installed on the Enterprise Manager host
• No Veeam B&R console / Veeam PowerShell module on the Enterprise Manager host

Configuration

• backup.framework.enabled = true
• backup.framework.provider.plugin = veeam
• backup.plugin.veeam.url = https://enterprise-manager-host:9398/api/
• backup.plugin.veeam.validate.ssl = false
• integration.veeam.control.enabled = false

Steps to reproduce

1. Install Veeam Backup Enterprise Manager on a dedicated server, separate from the Veeam Backup & Replication server(s), and add the backup server(s) to it.
2. Install and enable OpenSSH Server on the Enterprise Manager host and create a service account with the Portal Administrator role in Enterprise Manager.
3. Configure the backup.plugin.veeam.* settings in CloudStack with the Enterprise Manager URL and the service account.
4. In Veeam B&R, create a template backup job (targeting an unused vCenter tag, no schedule).
5. In CloudStack, import the template job as a Backup Offering. This works and the jobs are listed.
6. Assign a VMware instance to the imported Backup Offering.

Expected result

The template job is cloned, the instance is added to the cloned job, and the instance is assigned to the Backup Offering.

Actual result

Failed to add Instance to backup offering (errorcode 530)

• The REST connection to Enterprise Manager (TCP 9398) works: jobs are listed and the offering can be imported.
• The SSH session to the Enterprise Manager host is accepted (OpenSSH log: Accepted password for <service-account>).
• The PowerShell command sent over SSH fails, because the Veeam B&R PowerShell module is not present on the Enterprise Manager host. cloneVeeamJob cannot get the repository of the template job and the assignment fails.

Relevant management server log:

ERROR [c.c.u.s.SshHelper] SSH execution of command PowerShell Import-Module Veeam.Backup.PowerShell -WarningAction SilentlyContinue;$ProgressPreference='SilentlyContinue';$Job = Get-VBRJob -name 'CS-Template-Test';$Job.GetBackupTargetRepository() ^| select Name ^| Format-List has an error status code in return. Result output: Import-Module : The specified module 'Veeam.Backup.PowerShell' was not loaded because no valid module file was found in any module directory.
Get-VBRJob : The term 'Get-VBRJob' is not recognized as the name of a cmdlet, function, script file, or operable program.
You cannot call a method on a null-valued expression.

WARN  [o.a.c.b.v.VeeamClient] Exception caught while trying to clone Veeam job: com.cloud.utils.exception.CloudRuntimeException: Failed to get Repository Name from Job [name: CS-Template-Test].
        at org.apache.cloudstack.backup.veeam.VeeamClient.getRepositoryNameFromJob(VeeamClient.java:434)
        at org.apache.cloudstack.backup.veeam.VeeamClient.listBackupRepository(VeeamClient.java:411)
        at org.apache.cloudstack.backup.veeam.VeeamClient.cloneVeeamJob(VeeamClient.java:524)
        at org.apache.cloudstack.backup.VeeamBackupProvider.assignVMToBackupOffering(VeeamBackupProvider.java:171)

ERROR [o.a.c.b.VeeamBackupProvider] Failed to clone pre-defined Veeam job (backup offering) for backup offering [...] but will check the list of jobs again if it was eventually succeeded.
DEBUG [o.a.c.b.v.VeeamClient] Response received in GET request is: HTTP/1.1 200 OK ... for URL: [https://<enterprise-manager-host>:9398/api//jobs]
ERROR [o.a.c.b.BackupManagerImpl] Failed to assign Instance [...] to the Backup Offering [...] using provider [name: veeam, class: VeeamBackupProvider], due to: [Failed to assign the Instance to the Backup Offering, please try removing the assignment and try again.]

Proposed improvements

1. Allow the SSH / PowerShell target to be configured separately from the Enterprise Manager REST URL (for example a new setting such as backup.plugin.veeam.powershell.host), or use the Veeam B&R REST API for the operations that currently need PowerShell (e.g. reading the repository of a job and cloning a job).
2. Document clearly that the current implementation requires Enterprise Manager and Veeam B&R (with its PowerShell module) on the same host.
3. Confirm support for Veeam Backup & Replication v13. The plugin documentation mentions testing with versions 11 and 12 only.
4. Return a more specific error message when the PowerShell command fails (e.g. "Veeam PowerShell module not found on "), instead of the generic Failed to add Instance to backup offering.