Wait for Screenshot to Match
Los mantenedores suelen responder en 1 día
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 45/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Estancado
- Stack tecnológico
- typescript
- Área
- testing
Línea de trabajo
Comienza con los matchers de capturas de pantalla en @wdio/visual-service/dist/matcher.js y las constantes de timeout e intervalo en expect-webdriverio/src/constants.ts. Compara el flujo de waitUntil del prototipo con todos los matchers de capturas de pantalla y, después, verifica los valores configurables de timeout e intervalo, las sobrescrituras por llamada y la captura de la baseline cuando la espera agota el tiempo de espera.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Proposal
I'd like screenshot matchers to be re-executed until the screenshot matches.
Right now, one screenshot is taken and if it does not match, the test fails.
It may take a while until the browser/element is in the state where I want to take the screenshot.
It is not always useful/possible to wait for other factors that may indicate the page is in the desired state (i.e. existence of a css-class).
I'd like toMatchElementSnapshot and the others to be re-executed (within a configurable timeout) until the screenshot matches.
Prototype
I prototyped a solution where I locally changed toMatchElementSnapshot:
// @file ./node_modules/@wdio/visual-service/dist/matcher.js:97
export async function toMatchElementSnapshot(element, tag, expectedResultOrOptions, optionsOrUndefined) {
const { expectedResult, options } = parseMatcherParams(tag, expectedResultOrOptions, optionsOrUndefined);
const browser = getBrowserObject(await element);
let compared;
await browser.waitUntil(async () => {
const result = await browser.checkElement(await element, tag, options);
compared = compareResult(result, expectedResult || DEFAULT_EXPECTED_RESULT);
return compared.pass;
});
return compared;
}
This has the desired results.
Custom-Matcher (not possible)
I tried to write a custom matcher:
// @file ./extensions/expect_toMatchElementSnapshotSoon.ts
import { WdioCheckElementMethodOptions } from '@wdio/visual-service/dist/types';
import { toMatchElementSnapshot } from '@wdio/visual-service/dist/matcher'; // not possible
export async function toMatchElementSnapshotSoon (
element: WebdriverIO.Element,
tag: string,
expectedResultOrOptions?: number | ExpectWebdriverIO.PartialMatcher,
optionsOrUndefined?: WdioCheckElementMethodOptions
) {
let compared;
await element.waitUntil(async () => {
compared = await toMatchElementSnapshot(element, tag, expectedResultOrOptions, optionsOrUndefined);
return compared.pass;
});
return compared;
}
But when I want to run the tests, @wdio/config:ConfigParser fails:
2024-12-17T08:52:30.261Z ERROR @wdio/config:ConfigParser: Failed loading configuration file: file:///C:/dev/wdio-visual/mocha-wdio.conf.ts: Package subpath './dist/matcher' is not defined by "exports" in C:\dev\wm-gti-uts\wdio\node_modules\@wdio\visual-service\package.json imported from C:\dev\wm-gti-uts\wdio\extensions\expect_toMatchElementSnapshotSoon.ts
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './dist/matcher' is not defined by "exports" in C:\dev\wm-gti-uts\wdio\node_modules\@wdio\visual-service\package.json imported from C:\dev\wm-gti-uts\wdio\extensions\expect_toMatchElementSnapshotSoon.ts
at exportsNotFound (node:internal/modules/esm/resolve:304:10)
at packageExportsResolve (node:internal/modules/esm/resolve:651:9)
at packageResolve (node:internal/modules/esm/resolve:837:14)
at moduleResolve (node:internal/modules/esm/resolve:927:18)
at defaultResolve (node:internal/modules/esm/resolve:1169:11)
at nextResolve (node:internal/modules/esm/hooks:866:28)
at resolveBase (file:///C:/dev/wdio-visual/node_modules/tsx/dist/esm/index.mjs?1734425546202:2:3212)
at resolveDirectory (file:///C:/dev/wdio-visual/node_modules/tsx/dist/esm/index.mjs?1734425546202:2:3584)
at resolveTsPaths (file:///C:/dev/wdio-visual/node_modules/tsx/dist/esm/index.mjs?1734425546202:2:4073)
at resolve (file:///C:/dev/wdio-visual/node_modules/tsx/dist/esm/index.mjs?1734425546202:2:4447) {
code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
}
This is how I setup the custom matcher
import { toMatchElementSnapshotSoon } from "./extensions/expect_toMatchElementSnapshotSoon";
export const config: WebdriverIO.Config = {
// [...]
async before(): Promise<void> {
// const {toMatchElementSnapshotSoon} = await import('./extensions/expect_toMatchElementSnapshotSoon');
if (global.expect.expect !== undefined) { // Temporary workaround. See https://github.com/webdriverio/expect-webdriverio/issues/835
global.expect = global.expect.expect;
}
expect.extend({
toMatchElementSnapshotSoon,
});
},
}
btw. before() seems not to be awaited. I initially tried the dynamic import, but neither the code in the module, nor the code
after the await is executed.
Implementation
I came to the conclusion that this feature needs to be added to @wdio/visual-service.
The wait/timeout and interval should be the same as for the matchers in expect-webdriverio.
It also should be possible to override these per call to toMatchElementSnapshot() etc.
Baseline screenshots should be taken when the test times out - if the page is not in the desired state at that point, the test will fail in the future anyway.
IMO toMatchElementSnapshot and the others can safely be changed to retry the match. I can not think of cases where tests would start to fail or to have false-positives.
- Lenguaje dominante
- TypeScript
- Estrellas
- 156
- Forks
- 62
- Merge medio
- 2 h 3 min
- PR fusionados (30 d)
- 110
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de webdriverio/visual-testing
-
Proposal: a noise-tolerant compare mode with pixelmatch 8's windowSizePosiblemente ocupada @plum117 la tomó hace 1 día. Abiertoenhancement
webdriverio/visual-testing#1311 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
bug: the visual service fails on Android Chrome through Appium 3 in the default WebDriver BiDi session (workaround: `wdio:enforceWebDriverClassic`)Posiblemente ocupada @plum117 la tomó hace 1 día. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
webdriverio/visual-testing#1232 · 2 comentarios · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
feat: support element screenshots of elements in a frame in WebDriver BiDi sessions (v9 `switchFrame()`, v10 `context.frame()`)Posiblemente ocupada @plum117 la tomó hace 1 día. Abierto
Dificultad 5/5 Más de una semana Aptitud para principiantes 50/100
webdriverio/visual-testing#1228 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
bug help wanted
Dificultad 4/5 3-5 días Aptitud para principiantes 25/100
webdriverio/visual-testing#232 · 3 comentarios ·
Los mantenedores suelen responder en 1 día
-
Using checkFullPageScreen method, capturing images with duplicate viewport and is not scrolling to the bottom of the pagePosiblemente ocupada Un pull request vinculado a esta issue está abierto o ya se fusionó. Abiertohelp wanted
Dificultad 4/5 3-5 días Aptitud para principiantes 35/100
webdriverio/visual-testing#125 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de webdriverio/visual-testing
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
NousResearch/hermes-agent#136483 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
Los mantenedores suelen responder en 1 día
-
Tool errors containing cycles or BigInt crash getErrorMessage and replace the original failureAbiertofactory-active factory-automatic task-bug-reproduction-success task-identify-harness-labels-done task-identify-issue-type-done
Dificultad 2/5 1-3 horas Aptitud para principiantes 62/100
vercel/ai#22796 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
-
[Bug]: Web chat input doesn't regain focus after a reply finishesPosiblemente ocupada @GaijinSystems la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
zeroclaw-labs/zeroclaw#11658 ·
Los mantenedores suelen responder en 2 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
babylonlabs-io/babylon-toolkit#2711 ·
Los mantenedores suelen responder en 1 día