webdriverio/webdriverio

[🐛 Bug]: WebdriverIO toMatchSnapshot does not fail tests on mocha test retries

Open

#12.882 aperta il 16 mag 2024

Vedi su GitHub
 (7 commenti) (0 reazioni) (0 assegnatari)JavaScript (1793 fork)batch import
Bug 🐛help wanted

Metriche repository

Star
 (6029 star)
Metriche merge PR
 (Merge medio 15g 15h) (48 PR mergiate in 30 g)

Descrizione

Have you read the Contributing Guidelines on issues?

WebdriverIO Version

8.36.1

Node.js Version

v20.9.0

Mode

WDIO Testrunner

Which capabilities are you using?

No response

What happened?

When utilizing the toMatchSnapshot expectation in WebdriverIO, it is observed that the test run does not fail as expected when the snapshot does not match the current state of the tested element property.

This behavior is inconsistent with the intended functionality of snapshot testing, where a test should fail if there are discrepancies between the actual output and the stored snapshot.

The test run passes, despite the discrepancy between the snapshot and the current output, with a console log:

Snapshot Summary: 1 snapshot(s) unmatched.

What is your expected behavior?

The test run should fail, indicating that the snapshot does not match the current property.

How to reproduce the bug.

  1. Write a WebdriverIO test case that includes the toMatchSnapshot expectation:
describe('Snapshot testing', () => {
  it('should adequately assert texts', async () => {
    await browser.url('https://the-internet.herokuapp.com/');
    await expect($('ul').getProperty('textContent')).toMatchSnapshot('linkList');
  });
});
  1. Run the test
  2. Change the snapshot text
  3. Run the test again

Relevant log output

[chrome 124.0.6367.208 windows #0-0] » \test\specs\snapshot.spec.ts
[chrome 124.0.6367.208 windows #0-0] Snapshot testing
[chrome 124.0.6367.208 windows #0-0]    ✓ should adequately assert texts (1 retries)
[chrome 124.0.6367.208 windows #0-0]
[chrome 124.0.6367.208 windows #0-0] 1 passing (2.2s)


Snapshot Summary:
1 snapshot(s) unmatched.

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues

Guida contributor