[BUG] API deployment to UAT (next env) fails for new operations after upgrading to v6.0.1.6

Bắt đầu với đường dẫn publisher v6.0.1.6 trong tools/code/publisher/ApiOperationPolicy.cs, tại khoảng dòng 181, sau đó kiểm tra common/ApiOperationPolicy.cs tại khoảng dòng 132 và lỗi request đã được tái hiện. So sánh thứ tự triển khai các operation hiện có và operation mới với v5.1.4, sử dụng các bước tái hiện Dev-to-UAT được liệt kê. Được xem là hoàn tất khi các operation mới và policy của chúng được publish thành công lên UAT mà không gặp lỗi validation của Azure API Management.

Release version

v6.0.1.6

Describe the bug

I attempted to publish an API from Dev to UAT that included several new operations. The API already existed in UAT with a few operations previously deployed. I had added new operations in Dev and wanted to push those changes through.

Originally, I published this API to UAT using version 5.1.4 of the extractor/publisher pipeline. To stay current, I upgraded the pipeline to v6.0.1.6. The extractor ran successfully, and the publisher worked fine when targeting Dev. However, when publishing to UAT, everything worked until it tried to deploy the new operations.

The existing operations deployed without issue, but the new ones failed with the following error:

info: publisher[0]
Putting policy policy for operation scrubbed-already-there in API scrubbed-v1...
info: publisher[0]
Putting policy policy for operation scrubbed-new in API scrubbed-v1...
crit: publisher[0]
Application failed.
System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/scrubbed/providers/Microsoft.ApiManagement/service/apimuatscrubbed/apis/scrubbed-v1/operations/scrubbed-new/policies/policy?api-version=2023-09-01-preview failed with status code 400. Content is '{"error":{"code":"ValidationError","message":"Entity with specified identifier not found","details":null}}'.
at common.HttpPipelineExtensions.<>c__DisplayClass11_0.b__0(Response response) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at LanguageExt.Either.Left2.Match[B](Func2 Left, Func2 Right) at LanguageExt.Either2.IfLeft(Func`2 leftMap)
at common.HttpPipelineExtensions.PutContent(HttpPipeline pipeline, Uri uri, BinaryData content, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at common.ApiOperationPolicyModule.PutDto(ApiOperationPolicyUri uri, ApiOperationPolicyDto dto, HttpPipeline pipeline, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/ApiOperationPolicy.cs:line 132
at publisher.ApiOperationPolicyModule.<>c__DisplayClass11_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiOperationPolicy.cs:line 181

The error message I’m seeing might not reflect the actual root cause — I’m not sure why it would attempt to reach that URL when the operation hasn’t even been deployed there yet.

The spec file is relatively short, and the policies are simple. There are no validation errors, and all namedValues referenced exist in the target environment.

I reviewed issues #329 , #339 , #513 , and #553 and tried all suggested workarounds except manually exporting the API spec from Dev and importing it into UAT — which defeats the purpose of using DevOps automation.

I believe #513 and @daviian are on the right track, and the issue discussed there may still be unresolved.

As a test, I reverted the extractor and publisher pipelines back to v5.1.4, re-extracted the API and all related resources, and published again. This time, everything worked as expected.

For context, my API is a REST API with about six operations and fairly basic policies.

Expected behavior

The publisher should have published the new operations to the api without error.

Actual behavior

Originally, I published this API to UAT using version 5.1.4 of the extractor/publisher pipeline. To stay current, I upgraded the pipeline to v6.0.1.6. The extractor ran successfully, and the publisher worked fine when targeting Dev. However, when publishing to UAT, everything worked until it tried to deploy the new operations.

The existing operations deployed without issue, but the new ones failed with the following error:

info: publisher[0]
Putting policy policy for operation scrubbed-already-there in API scrubbed-v1...
info: publisher[0]
Putting policy policy for operation scrubbed-new in API scrubbed-v1...
crit: publisher[0]
Application failed.
System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/scrubbed/providers/Microsoft.ApiManagement/service/apimuatscrubbed/apis/scrubbed-v1/operations/scrubbed-new/policies/policy?api-version=2023-09-01-preview failed with status code 400. Content is '{"error":{"code":"ValidationError","message":"Entity with specified identifier not found","details":null}}'.
at common.HttpPipelineExtensions.<>c__DisplayClass11_0.b__0(Response response) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at LanguageExt.Either.Left2.Match[B](Func2 Left, Func2 Right) at LanguageExt.Either2.IfLeft(Func`2 leftMap)
at common.HttpPipelineExtensions.PutContent(HttpPipeline pipeline, Uri uri, BinaryData content, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/Http.cs:line 173
at common.ApiOperationPolicyModule.PutDto(ApiOperationPolicyUri uri, ApiOperationPolicyDto dto, HttpPipeline pipeline, CancellationToken cancellationToken) in /home/runner/work/apiops/apiops/tools/code/common/ApiOperationPolicy.cs:line 132
at publisher.ApiOperationPolicyModule.<>c__DisplayClass11_0.<b__0>d.MoveNext() in /home/runner/work/apiops/apiops/tools/code/publisher/ApiOperationPolicy.cs:line 181

The error message I’m seeing might not reflect the actual root cause — I’m not sure why it would attempt to reach that URL when the operation hasn’t even been deployed there yet.

The spec file is relatively short, and the policies are simple. There are no validation errors, and all namedValues referenced exist in the target environment.

When reverting back to v5.1.4 everything worked fine.

Reproduction Steps

1. Create a basic REST api in lower env (dev)
2. Create a few operations in that api
3. Extract that api and anything that pertains to it using the pipeline (v6.0.1.6)
4. (may want to skip to step 8 but continuing here might trigger a failure too) Publish that artifact to the next env (uat)
5. This should fail but in my case I used an older version of the extractor and publisher (v5.1.4) the first time they went to uat so the api and 2 operations were already there.
6. If it didnt fail already add two new operations in dev for that api.
7. extract and publish again and it should fail only on the new operations
8. To set this up better you may want to manually extract the spec file and import it into uat instead of running the publisher in step 4. then start at step 6 once dev and uat line up.