Better "expected results" in ADA

Open
#668 0 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

Assessment

This issue has not been assessed yet.

Description

A common failure in ADA is the result of poor result formatting. In my experience we can significantly improve on this by having the generation use a regular expression for validation rather than the similarity. This is because the system will often use truncated results (which is good from a readability perspective). For example:

{
    "agentPoolProfiles": [
        {
            "name": "nodepool1",
            "count": 1,
            "vmSize": "Standard_DS2_v2",
            "maxPods": 110,
            "osType": "Linux"
        }
    ],
    "dnsPrefix": "AKSClusterxxx-dns",
    "provisioningState": "Succeeded",
    "resourceGroup": "AKSLabResourceGroupxxx",
    "name": "AKSClusterxxx",
    "type": "Microsoft.ContainerService/ManagedClusters",
    ...
}

However, this is not valid JSON and thus the validation step fails.

In this case, and in many like it, what we are actually looking for is "provisioningState": "Succeeded". Therefore for responses like this one we can use:

<!-- expected_similarity=.*Succeeded -->

On a similar note, it seems the tool gives a default similarity value of 0.3 which, for the most part, is too low to be reliable.

Dominant language
Python
Stars
13
Forks
26
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MicrosoftDocs/executable-docs

All issues in MicrosoftDocs/executable-docs

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.