Visual Assertions in UI Tests

Automation Framework Customization: Part-1 | Expertise: Chūnin

In this article, we will look at how we can create our own custom Visual Assertion class to compare screenshots and optimize it further to visually validate the correctness and appearance of any given UI components.

Visual Validation of UI Components

But first, let's understand why our UI tests fail to give us the confidence that we all are longing for. This is because typical UI tests either →

• Renders the component in a virtual DOM (i.e. unit/component tests)
• Load the entire page including the component in a real browser (acceptance/e2e)

And in both aforementioned cases, we end up checking whether the given component has rendered correctly in a DOM or located (loaded) in a browser; however, we still cannot verify if a given component is correctly visible on UI “as it should be” and NOT spilt over, distorted, overlapped or simply incorrect in any way.

So does this mean, no matter how good our UI tests are, no one in this deciduous world can guarantee that each and every component of a given UI can be verified by just running such automated UI tests?
And thus, we will always need the expertise of a mere earthling to use their human eyes to give us that, “everything looks good” confidence.

Well, the above thought process is simply bullshit!

Let's solve this issue by Visually Asserting Screenshots of WebElements in our UI tests. We will primarily use Shutterbug and AssertJ libraries to perform automated visual validations.

Prerequisites:
• UI Test Project using Selenium WebDriver

Step 1: Add the Shutterbug and AssertJ dependencies to your build file.

For Maven: update pom.xml

<dependency>
    <groupId>com.assertthat</groupId>
    <artifactId>selenium-shutterbug</artifactId>
    <version>1.6</version>
</dependency>
<dependency>
    <groupId>org.assertj</groupId>
    <artifactId>assertj-core</artifactId>
    <version>3.23.1</version>
</dependency>

For Gradle: update build.gradle

implementation 'com.assertthat:selenium-shutterbug:1.6.0'
implementation 'org.assertj:assertj-core:3.23.1'

Step 2: Create an AssertSnapshot class

AssertSnapshot.java

Note: You may want to move both methods takeWebElementScreenshot() and getFilePathForFile() to your Test Starter class like BaseTest which is extended by all other tests in the project.

Step 3: Add Visual Assertions to your UI Tests

Let’s create a test class which has two test methods.

• Test 1: The first test will take a screenshot of the Google Search component at runtime and compare it with an expected image which is correct so it will pass.
• The second test, however, compares the runtime screenshot with an incorrect image to show the test failing.

GoogleSearchTest.java

Note: Before comparing a screenshot, you wil need to create an expected screenshot. You can do this by simply executing takeWebElementScreenshot() first without performing any Visual Assertion.

Finally below are two expected images we will use in tests accordingly.

Correct Expected Image used in Test-1

Incorrect Expected Image used in Test-2

Step 4: Now run your tests and verify the functionality.

As expected Test-1 will pass with the correct expected image and Test-2 will fail with the incorrect expected image. Below are the test-run results displayed in IDE.

Testrun Results in IDE

Our failed Visual Assertion will also create a new Image with the differences highlighted in red as below.

Resultant Image Highlighting Differences in Red

Step 5: Configure Browser Size

Your visual assertions may still fail if the resolutions of both the expected and actual images differ. One way to resolve this is by setting the browser window to a default size just after instantiating Selenium WebDriver.

driver.manage().window().setSize(new Dimension(1920, 1080));

Step 6: Calibrate Deviation Rate

Shutterbug library allows us to set the deviation rate which represents the percentage of image difference between the expected and actual images.

However, if you set this to 0.0, it will fail the test for even minuscule image differences. Therefore adjust this rate to a value which works as per your requirements. (like 0.1 or 0.001)

Links:

The above illustration is a tiny part of my personal project Sharingan; however, I have added the minimum required code to my Github project which is used in this article.

GitHub - suryajit7/my-blog-topics: This is just a test project to showcase sample code for ideas…