[Feature]: Guaranteed testing against latest published version of Canvas Apps
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- csharp
- Domain
- testing-qa
Research direction
No files, tests, or entry points are named. Start by locating Test Engine's Canvas App launch and version-handling flow, then determine how it could compare the running app with the latest published version and wait or reload; done means tests consistently run against the version intended for deployment.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
A long-standing issue with Canvas Apps is that when users open an app that was very recently re-published, they may actually get the previous version instead of the latest version. For example, if version 1 was published yesterday, and version 2 was published 5 minutes ago, launching the app might open version 1 instead of version 2.
This issue affects Test Engine just as it does regular users.
Although this is not an issue with Test Engine itself, it is not acceptable for this scenario to exist in any context where Test Engine is being used.
- The tests defined in Test Engine's test plan will be defined with version N in mind, and may not work as expected on version N-1, producing false positives or false negatives
- Test Engine may be used in a CI/CD pipeline, and the test results (pass vs fail) may be critical to deciding if export and/or deployment should proceed, so it is vital that Test Engine is testing (100% of the time) the version of the Canvas App that is being deployed (most recently published, with no delay)
Describe the solution you'd like
- "Fix" the player so if a newer version of the app is published (or in the process of being published) it waits for publishing to finish and loads that new version
- Perhaps this could be controlled by a query string parameter so it doesn't affect business users in production, but honestly I've heard many users in production complain that they aren't seeing the latest version of an app, several minutes or even hours after publish/import
- Btw, I do understand this is a larger ask, probably beyond the control of the Test Engine team, but I do think the Test Engine team should be championing a solution for this problem
OR
- Have the Test Engine compare the version of the currently running application against the last recorded version of the application, and if they do not match then wait/reload until the two versions match and then proceed with testing
Describe alternatives you've considered
Definitely not ideal, but...
- Before every publish, add expected version number to a text input control (cannot use variables because of #624)
- Use connector get the latest recorded version of the app in Dataverse
- In another text input control, set the default value of another text input control to the latest recorded version
- In test case, compare the values of the two text controls, and fail if they do not
Additional context?
No response
- Dominant language
- C#
- Stars
- 150
- Forks
- 55
- Avg merge
- 5d 23h
- Merged PRs (30d)
- 4
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/PowerApps-TestEngine
-
new
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
new
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
new
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
question
Difficulty 5/5 Over a week Newbie friendliness 20/100
All issues in microsoft/PowerApps-TestEngine
Similar issues
-
effort:S P3 refactor
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
nightscout/nocturne#1532 ·
-
core dependencies
Difficulty 1/5 Under an hour Newbie friendliness 80/100
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
C#/.NET Roslyn LSP Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
DotNetNext/SqlSugar#1458 ·