Sauce Visual Testing
Get clear and instant visibility into the impact of code changes on your UI across browsers, devices, and operating systems with our automated visual testing solution, which generates side-by-side comparisons of your UI versions with DOM diffs inspection.
Know exactly where and what has changed so that you can deliver a consistent and intuitive user experience.
Sauce Visual Testing stands for:
Faster Test Execution
- Purpose-built within the world’s best test infrastructure
- Fewer back-and-forth commands between client and server
- Based on efficient OSS framework
- No need to down/upload images
Less Complexity
- Less code
- Less maintenance
- Less flakiness
What You'll Need
- A Sauce Labs account (Log in or sign up for a free trial license) to access Visual Testing.
- Your Sauce Labs Username and Access Key.
Workflows
Sauce Visual testing consists of two main workflows: Test execution and review. Both workflows need to implemented, but they may performed by different people or teams. Read below to learn more about the workflows.
Test Execution
This workflow is responsible for generating snapshots (screenshots) of the website or app under test. Users new to visual testing often have an existing test suite in WebdriverIO, Cypress or a similar framework. With Sauce Visual testing existing test suites can be enhanced with a few extra "take a snapshot" statements and that is all it takes to implement this workflow.
Review
This workflow is responsible for approving or rejecting detected changes between a snapshot and a baseline. Users will be available to review changes and make the baseline evolve for accepted changes.
Concepts
Sauce Visual Testing comes with its set of concepts that will be defined here.
Definitions
Snapshot
A snapshot is an image that will be compared by Visual Testing. For example: the screenshot of your website or mobile application.
Build
A build groups multiple snapshots. E.g. to be able to view all snapshots taken during the execution of your test suite in CI or locally.
Build Status
A build has a status that represents its current state:
| Status | Description |
|---|---|
Accepted | Changes have been detected and they have been accepted |
No changes | No difference has been detected between snapshots and the matching baselines |
Running | Build is still running |
Queued | Snapshots are waiting to be compared against baselines |
For review | Changes have been detected and and action is needed |
Errored | An error has occurred |
Rejected | Changes have been detected and they have been rejected |
No Snapshots | There is no snapshot in this build |
Baseline
A baseline is what a snapshot is compared to.
Diff
A diff represents the differences that have been detected between a snapshot and its matching baseline.
Diff Status
A diff has a status that represents its current state:
| Status | Description |
|---|---|
No Changes | No difference has been detected between this snapshot and the matching baseline |
For Review | Changes have been detected and an action is needed (meaning snapshots without baselines or with a visual difference) |
Accepted | Changes have been detected and they have been accepted |
Rejected | Changes have been detected and they have been rejected |
Baseline Matching
Every integration generates snapshots and compares them against a baseline snapshot to detect differences. The following snapshot metadata keys are used to determine the appropriate baseline:
| Property | Type | Description |
|---|---|---|
name | string | The name of the snapshot which will be set when the sauceVisualCheck() is called. See each language binding. |
testName | string | The name of the test where the snapshot has been taken (only with Cypress, WebdriverIO, and Storybook BETA). This will automatically determined by the language binding. |
suiteName | string | The name of the suite where the snapshot has been taken (only with Cypress, WebdriverIO, and Storybook BETA). This will automatically determined by the language binding. |
browser | string | The browser used to take the snapshot. This will automatically be determined by Sauce Visual. |
operatingSystem | string | The operating system used to take the snapshot. This will automatically be determined by Sauce Visual. |
operatingSystemVersion | string | The operating system version used to take the snapshot. This will automatically be determined by Sauce Visual. |
project | string | The project name. This can be set during service initiation, for more information see the language bindings. |
branch | string | The branch name. This can be set during service initiation, for more information see the language bindings. |
device | string | The device name. This will automatically be determine by Sauce Visual. |
If multiple baselines match, the most recent one is selected for diff computation.
Note that not all properties are exposed on all testing frameworks. In these cases, a default value (0, null or empty string) is used.
The matching process happens as part of the snapshot creation (createSnapshot in the API).
This means, that a baseline can only be considered for a diff if it existed before the createSnapshot call.
Integrations
Sauce Visual provides integrations with different kind of frameworks:
- E2E Testing allows to navigate a website and capture snapshots any time this is needed.
- Component Testing allows to test extensively all your components.