[BUG] Publishing API fails

Start in tools/code/publisher/ApiDiagnostic.cs at the diagnostic publishing flow and compare it with the values from configuration.prod.yaml and apimartifacts/apis/sendemail-v1/diagnostics/applicationinsights/diagnosticsInformation.json. Trace which loggerId is selected for the API diagnostic, then verify that publishing uses the configured value instead of the extracted value and no longer fails with the reported Azure validation error.

Release version

APIOps Toolkit for Azure APIM v6.0.1.4

Describe the bug

Facing issues while Publising API using GitHub Workflows. Everything gets Published fine and fails while adding diagnostic applicationinsights to API.

I have created "configuration.prod.yaml" with required loggerid. Below is my config file:

`apimServiceName: PRACTICE-APIM-PROD
namedValues:

• name: commonutilities-backend-uri
  properties:
  displayName: commonutilities-backend-uri
  value: "/workflows/f9ed02c791d74149be5fb8be4ec1d005/triggers/When_a_HTTP_request_is_received/paths/invoke?api-version=2016-10-01\u0026sp=%2Ftriggers%2FWhen_a_HTTP_request_is_received%2Frun\u0026sv=1.0\u0026sig="
• name: commonutilities-backend-uri-key
  properties:
  displayName: commonutilities-backend-uri-key
  keyVault:
  secretIdentifier: "https://practice-kvault-prod.vault.azure.net/secrets/commonutilities-backend-uri-key"
  backends:
• name: commonutilities-backend-url
  properties:
  url: "https://prod-43.eastus.logic.azure.com:443"
  diagnostics:
• name: applicationinsights
  properties:
  loggerId: "/subscriptions/be0a5402-e682-4591-9208-a285c33b2699/resourceGroups/Practice-Prod/providers/Microsoft.ApiManagement/service/PRACTICE-APIM-PROD/loggers/practice-appi-prod"
• name: azuremonitor
  properties:
  loggerId: "/subscriptions/be0a5402-e682-4591-9208-a285c33b2699/resourceGroups/Practice-Prod/providers/Microsoft.ApiManagement/service/PRACTICE-APIM-PROD/loggers/azuremonitor"
  apis:
• name: sendemail-v1
  diagnostics:
  • name: applicationinsights
    properties:
    verbosity: Error
    loggerId: "/subscriptions/be0a5402-e682-4591-9208-a285c33b2699/resourceGroups/Practice-Prod/providers/Microsoft.ApiManagement/service/PRACTICE-APIM-PROD/loggers/practice-appi-prod"`

But instead of overriding LoggerID from this configuration, it is trying to use the same details from Extracted Json from "apimartifacts/apis/sendemail-v1/diagnostics/applicationinsights/diagnosticsInformation.json" and failed with below error:

info: publisher[0] Adding diagnostic applicationinsights to API sendemail-v1... crit: publisher[0] Application failed. System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/Practice-Prod/providers/Microsoft.ApiManagement/service/PRACTICE-APIM-PROD/apis/sendemail-v1/diagnostics/applicationinsights?api-version=2023-09-01-preview failed with status code 400. Content is '{"error":{"code":"ValidationError","message":"One or more fields contain incorrect values:","details":[{"code":"ValidationError","target":"loggerId","message":"Logger Id 'practice-appi-dev' does not exist"}]}}'. at common.HttpPipelineExtensions.<>c__DisplayClass11_0.<PutContent>b__0(Response response) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173 at LanguageExt.Either.Left2.Match[B](Func2 Left, Func2 Right)
at LanguageExt.Either2.IfLeft(Func2 leftMap)
at common.HttpPipelineExtensions.PutContent(HttpPipeline pipeline, Uri uri, BinaryData content, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at common.ApiDiagnosticModule.PutDto(ApiDiagnosticUri uri, ApiDiagnosticDto dto, HttpPipeline pipeline, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/ApiDiagnostic.cs:line 274
at publisher.ApiDiagnosticModule.<>c__DisplayClass11_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 176
--- End of stack trace from previous location ---
at publisher.ApiDiagnosticModule.<>c__DisplayClass7_1.<b__1>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 122
--- End of stack trace from previous location ---
at common.OptionExtensions.IterTask[T](Option1 option, Func3 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Option.cs:line 22
at common.OptionExtensions.IterTask[T](Option1 option, Func2 action) in /home/runner/work/apiops/apiops/tools/code/common/Option.cs:line 16
at publisher.ApiDiagnosticModule.<>c__DisplayClass7_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 122
--- End of stack trace from previous location ---
at common.IEnumerableExtensions.<>c__DisplayClass5_02.<<IterParallel>b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 69 --- End of stack trace from previous location --- at System.Threading.Tasks.Parallel.<>c__531.<b__53_0>d.MoveNext()
--- End of stack trace from previous location ---
at common.IEnumerableExtensions.IterParallel[T](IEnumerable1 enumerable, Func3 action, Int32 maxDegreeOfParallelism, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 61
at common.IEnumerableExtensions.IterParallel[T](IEnumerable1 enumerable, Func3 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 47
at common.IEnumerableExtensions.IterParallel[T1,T2](IEnumerable1 enumerable, Func4 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 69
at publisher.ApiDiagnosticModule.<>c__DisplayClass1_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 54
--- End of stack trace from previous location ---
at publisher.AppModule.<>c__DisplayClass1_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/App.cs:line 180
--- End of stack trace from previous location ---
at common.HostingModule.RunApplication(IHost host) in /home/runner/work/apiops/apiops/tools/code/common/Hosting.cs:line 89
info: Microsoft.Hosting.Lifetime[0]
Application is shutting down...
Unhandled exception. System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/Practice-Prod/providers/Microsoft.ApiManagement/service/PRACTICE-APIM-PROD/apis/sendemail-v1/diagnostics/applicationinsights?api-version=2023-09-01-preview failed with status code 400. Content is '{"error":{"code":"ValidationError","message":"One or more fields contain incorrect values:","details":[{"code":"ValidationError","target":"loggerId","message":"Logger Id 'practice-appi-dev' does not exist"}]}}'.
at common.HttpPipelineExtensions.<>c__DisplayClass11_0.b__0(Response response) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at LanguageExt.Either.Left2.Match[B](Func2 Left, Func2 Right) at LanguageExt.Either2.IfLeft(Func2 leftMap) at common.HttpPipelineExtensions.PutContent(HttpPipeline pipeline, Uri uri, BinaryData content, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173 at common.ApiDiagnosticModule.PutDto(ApiDiagnosticUri uri, ApiDiagnosticDto dto, HttpPipeline pipeline, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/ApiDiagnostic.cs:line 274 at publisher.ApiDiagnosticModule.<>c__DisplayClass11_0.<<GetPutApiDiagnosticInApim>b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 176 --- End of stack trace from previous location --- at publisher.ApiDiagnosticModule.<>c__DisplayClass7_1.<<GetPutApiDiagnostic>b__1>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 122 --- End of stack trace from previous location --- at common.OptionExtensions.IterTask[T](Option1 option, Func3 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Option.cs:line 22 at common.OptionExtensions.IterTask[T](Option1 option, Func2 action) in /home/runner/work/apiops/apiops/tools/code/common/Option.cs:line 16 at publisher.ApiDiagnosticModule.<>c__DisplayClass7_0.<<GetPutApiDiagnostic>b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 122 --- End of stack trace from previous location --- at common.IEnumerableExtensions.<>c__DisplayClass5_02.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 69
--- End of stack trace from previous location ---
at System.Threading.Tasks.Parallel.<>c__531.<<ForEachAsync>b__53_0>d.MoveNext() --- End of stack trace from previous location --- at common.IEnumerableExtensions.IterParallel[T](IEnumerable1 enumerable, Func3 action, Int32 maxDegreeOfParallelism, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 61 at common.IEnumerableExtensions.IterParallel[T](IEnumerable1 enumerable, Func3 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 47 at common.IEnumerableExtensions.IterParallel[T1,T2](IEnumerable1 enumerable, Func4 action, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Enumerable.cs:line 69 at publisher.ApiDiagnosticModule.<>c__DisplayClass1_0.<<GetPutApiDiagnostics>b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiDiagnostic.cs:line 54 --- End of stack trace from previous location --- at publisher.AppModule.<>c__DisplayClass1_0.<<GetRunApplication>b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/App.cs:line [180](https://github.com/sangitapatra94/apim-aapiops/actions/runs/14420658670/job/40442782574#step:11:181) --- End of stack trace from previous location --- at common.HostingModule.RunApplication(IHost host) in /home/runner/work/apiops/apiops/tools/code/common/Hosting.cs:line 89 at common.HostingModule.RunHost(String[] arguments, String applicationName, Action1 configureRunApplication) in /home/runner/work/apiops/apiops/tools/code/common/Hosting.cs:line 24
at publisher.Program.Main(String[] arguments) in /home/runner/work/apiops/apiops/tools/code/publisher/Program.cs:line 10
at publisher.Program.

(String[] arguments)
Exception: /home/runner/work/_temp/d457408b-a9aa-4528-9cb6-c979977eaba6.ps1:41
Line |
41 | if ($LASTEXITCODE -ne 0) { throw "Running publisher failed."}
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Running publisher failed.
Error: Process completed with exit code 1.`

Expected behavior

It should have Override the LogerId from Publisher Config file while Adding Diagnostics to API.

Actual behavior

Instead of Overriding LoggerID from Publisher Config File it is using the Same LoggerId was extracted from Dev APIM.

Reproduction Steps

Extract the API from Lower Environment having ApplicationInsight Enabled in API setting. Then publish API to Higher Environment have LoggerId change configuration in Publisher Config YAML