Dry-run should perform client-side validation to catch failures that actual publish would hit
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- typescript
Research direction
Start by reading src/services/dry-run-reporter.ts, src/services/override-merger.ts, and src/lib/config-loader.ts, then compare tests/test-overrides/extract-dryrun.log with extract.log. Trace the final payload after override merging and identify how dry-run reports PUT operations. Done means the proposed client-side checks emit advisory [DRY RUN] WARN lines without blocking the report or changing its exit code.
Written by the indexing model from the issue text.
Description
Problem
The current --dry-run implementation only checks whether resources exist in the target APIM instance (GET to determine PUT vs create). It does not validate the merged payload, so dry-run reports success for resources that will fail during actual publish.
Evidence
Comparing tests/test-overrides/extract-dryrun.log (dry-run) vs tests/test-overrides/extract.log (actual publish), dry-run reported all resources as PUT while actual publish hit multiple HTTP 400 validation errors:
| Resource | Publish Error | Dry-run result |
|---|---|---|
namedvalue/src-nv-plain |
ValidationError — display name contains spaces/parens (invalid characters) |
✅ PUT |
namedvalue/src-nv-secret |
ValidationError — same display name issue |
✅ PUT |
namedvalue/src-nv-keyvault |
Managed identity clientId not found on target |
✅ PUT |
backend/src-backend-circuit-breaker |
failureCondition must not be empty |
✅ PUT |
backend/src-backend-function |
resourceId — "Value should represent absolute http URL" |
✅ PUT |
backend/src-backend-logicapp |
resourceId — same URL format issue |
✅ PUT |
logger/src-logger-appinsights |
Invalid instrumentation key | ✅ PUT |
logger/src-logger-eventhub |
HTTP 502/500 (connection string validation) | ✅ PUT |
diagnostic/applicationinsights |
Invalid loggerId reference |
✅ PUT |
diagnostic/azuremonitor |
Invalid loggerId reference |
✅ PUT |
Proposed Validations
The dry-run reporter should add client-side validation before reporting PUT. These checks should produce [DRY RUN] WARN lines without blocking the rest of the report. Suggested checks:
1. Named Value display name format
ARM rejects display names with spaces, parentheses, and other special characters. Validate against the pattern: ^[A-Za-z0-9._-]+$.
2. Resource ID format validation
resourceId fields on backends must be valid ARM resource IDs (starting with /subscriptions/). Currently, override values like bare paths pass through unchecked.
3. URL format validation
Backend url fields and API serviceUrl fields should be valid URLs with a scheme (https://, wss://, etc.).
4. Cross-resource reference validation (loggerId)
Diagnostics reference loggers by loggerId. Dry-run should verify the referenced logger exists either in the artifact set or in the target APIM instance. Same applies after override merging — if an override changes loggerId, validate the new value.
5. Circuit breaker rule completeness
ARM requires failureCondition when circuitBreaker.rules is specified. Validate that overrides don't produce partial circuit breaker configs.
6. Key Vault identity pre-check
When a named value references a Key Vault with identityClientId, verify that the identity is present on the target APIM service (the service's managed identities are available via the service resource GET).
7. Logger credential validation
Warn when logger credentials contain placeholder values (e.g., production-instrumentation-key-00000000) that are unlikely to be valid GUIDs or connection strings.
8. Override key validity
Warn when override properties don't match any known field in the resource's ARM schema (typo detection). For example, retryCondition vs the correct failureCondition nesting.
9. Dependency ordering warnings
When a resource depends on another resource that is itself invalid or skipped, warn that downstream resources will likely also fail.
Implementation Notes
- Validations should run after override merging, against the final payload that would be sent to ARM.
- Warnings should not change the exit code — they are advisory. Consider a
--strictflag that promotes warnings to errors. - These are client-side heuristics; ARM will always be the authoritative validator. The goal is to catch the most common mistakes early.
Related Files
src/services/dry-run-reporter.ts— main dry-run logicsrc/services/override-merger.ts— override mergingsrc/lib/config-loader.ts— override loading
- Dominant language
- TypeScript
- Stars
- 29
- Forks
- 10
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 22
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Azure/apiops-cli
-
type:question
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Azure/apiops-cli#277 ·
-
type:documentation
Difficulty 1/5 Under an hour Newbie friendliness 94/100
Azure/apiops-cli#250 ·
-
Documentation P2
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Azure/apiops-cli#24 · 1 comment ·
-
type:bug
Difficulty 4/5 3-5 days Newbie friendliness 55/100
Azure/apiops-cli#291 ·
-
type:bug
Difficulty 3/5 1-2 days Newbie friendliness 58/100
Azure/apiops-cli#276 ·
All issues in Azure/apiops-cli
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Eynzof/Hermes-CN-Desktop#610 ·
-
bug clawsweeper:linked-pr-open clawsweeper:needs-live-repro clawsweeper:no-new-fix-pr impact:message-loss issue-rating: 🐚 platinum hermit P2 regression
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
calcite-components needs triage refactor
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Esri/calcite-design-system#15203 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
fullcalendar/fullcalendar#8106 ·