[🐛 Bug]: WebdriverIO toMatchSnapshot does not fail tests on mocha test retries
#12.882 aperta il 16 mag 2024
Metriche repository
- Star
- (6029 star)
- Metriche merge PR
- (Merge medio 15g 15h) (48 PR mergiate in 30 g)
Descrizione
Have you read the Contributing Guidelines on issues?
- I have read the Contributing Guidelines on issues.
WebdriverIO Version
8.36.1
Node.js Version
v20.9.0
Mode
WDIO Testrunner
Which capabilities are you using?
No response
What happened?
When utilizing the toMatchSnapshot expectation in WebdriverIO, it is observed that the test run does not fail as expected when the snapshot does not match the current state of the tested element property.
This behavior is inconsistent with the intended functionality of snapshot testing, where a test should fail if there are discrepancies between the actual output and the stored snapshot.
The test run passes, despite the discrepancy between the snapshot and the current output, with a console log:
Snapshot Summary: 1 snapshot(s) unmatched.
What is your expected behavior?
The test run should fail, indicating that the snapshot does not match the current property.
How to reproduce the bug.
- Write a WebdriverIO test case that includes the toMatchSnapshot expectation:
describe('Snapshot testing', () => {
it('should adequately assert texts', async () => {
await browser.url('https://the-internet.herokuapp.com/');
await expect($('ul').getProperty('textContent')).toMatchSnapshot('linkList');
});
});
- Run the test
- Change the snapshot text
- Run the test again
Relevant log output
[chrome 124.0.6367.208 windows #0-0] » \test\specs\snapshot.spec.ts
[chrome 124.0.6367.208 windows #0-0] Snapshot testing
[chrome 124.0.6367.208 windows #0-0] ✓ should adequately assert texts (1 retries)
[chrome 124.0.6367.208 windows #0-0]
[chrome 124.0.6367.208 windows #0-0] 1 passing (2.2s)
Snapshot Summary:
1 snapshot(s) unmatched.
Code of Conduct
- I agree to follow this project's Code of Conduct
Is there an existing issue for this?
- I have searched the existing issues