Visual test driven development (VTDD) for responsive web pages using Galen

Responsive Web Design (RWD) makes a web page look good on all devices, but testing responsive web pages is a tedious task. So the challenge that we took up was to implement a strategy for testing responsive web pages with the test first approach. For some time my team was exploring possibilities to test page layouts on different browsers automatically. This led us to Galen test framework which is a layout and functional testing framework that has its own language for describing responsive page layouts and supports both Java and Javascript. Galen also promotes the test first attitude and uses real browsers to render the html and .css for all tests unlike the traditional TDD where tests are implemented in a white box.

Our Experiment:

To get going, we used a simple project. The setup was (Java + TestNG + Galen + Webdriver) and we used Maven to add the dependencies:

 <dependencies>

…
 <dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>6.7</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.galenframework</groupId>
    <artifactId>galen-java-support</artifactId>
    <version>2.1.2</version>
</dependency>
…

</dependencies>

Then we started writing tests for page layouts using Galen specs language, which can describe any layout including different screen sizes or even browsers. To run the test on different devices, we created a test setup.

Desktop setup:

@DataProvider(name = "desktop")
public Object[][] desktop() {
return new Object[][]{
        {new TestDevice("desktop", new Dimension(1030, 1200), asList("desktop"))}
    };
}

Tablet setup:

@DataProvider(name = "tablet-landscape")
 public Object[][] tabletLandscape() {
 return new Object[][]{
        {new TestDevice("tablet-landscape", new Dimension(1024, 768), asList("tablet-  landscape"))}
         };
}

And here's a sample test:

import com.galenframework.java.components.GalenTestBase;
import org.testng.annotations.Test;

public class LeaveFeedbackTest extends GalenTestBase {

private static String feedbackPageUrl = "/feedback/form/shop/hunde/hundefutter/bosch/123456";

protected LeaveFeedbackTest() throws IOException {
}
   @Test(dataProvider = "desktop")
public void LeaveFeedback_ShouldLook_Good_Desktop(TestDevice desktop) throws IOException, InterruptedException {
    load(feedbackPageUrl);
    checkLayout("/specs/product/productfeedback.spec", desktop.getTags());
       } 
}

When the tests are executed they produce a nice HTML report, which comes out of the box from Galen framework (using TestNG listener.)

 <configuration>
     <properties>
         <property>
         <name>listener</name>
         <value>com.galenframework.testng.GalenTestNgReportsListener</value>
         </property>
     </properties>
 </configuration>

So what do you test with Galen?

Let’s take an example:
The customer feedback page was recently redesigned so we thought why not try using the Galen test framework along with it to test the page layout. The new page was responsive and should look good and easy to use regardless of the device.

responsiveimage To check this new responsive design, layout tests were written as Galen specs which also serve as living documentation and explain how the page would look like on any device. Let’s write a simple test for the customer feedback page which could be part of the Galen Spec file.
We would like to check if the “Save button” is visible on the page for all devices and it should be 34-pixel below the feedback form. To test this we transform the above statement to a Galen specs.

save-button

= Save button =
@on *
    Savebutton:
        visible
        below feedback_form_box  34 px

The use of @on * annotation says that the test should run on all defined test devices.
The second test is to check if the rating stars are visible and exist inside the feedback form and are 15-pixel above the title field

ratingStars

= Rating stars =
   @on tablet-landscape, desktop
    ratingstars:
        visible
        inside feedback_form_box  5 px top
        above title-field  15 px

Finally all the tests for whole web page are transformed in a Galen spec file as shown below:

@objects
productPicture               id.productPicture
feedback-main-form           id feedbackForm
feedback-pagetitle           h1.pagetitle
product__rating__bar         id productratingbar
ratingstars                  span.icon-star
icon-star-outline            span.icon-star-outline
input-title                  id title
textarea-text                id text
input-name                   id name
input-email                  id email
button-submit                id email
stars-*                      id  icon-star
Savebutton                   id saveButton
feedback_form_box            .form__box

= ProductPicture =
@on tablet-portrait, tablet-small
    productPicture:
        absent
@on tablet-landscape, desktop
    productPicture:
        visible

= Feedback Form =
@on *
    feedback-main-form:
        contains product__rating__bar, feedback-pagetitle, product__rating__bar, input-title, textarea-text, input-name, input-email

= Rating stars =
   @on tablet-landscape, desktop
    ratingstars:
        visible
        inside feedback_form_box  5 px top
        above title-field  15 px

= Save button =
@on *
    Savebutton:
        visible
        below feedback_form_box  34 px           

Test execution and Galen Reports:

Test execution are done through TeamCity (build system) and the tests are triggered periodically every day. Every test execution generates a html report which is saved as artifacts. The report contains a summary of all layout tests that were executed and also has a heat map.

GalenReports detailedrport The heat map shows an overview of the layout validations and might be handy to see which areas are left with minimal validations or are not covered at all. A sample heat map of our customer feedback page example is shown below.
HeatMaps The Galen reports have brought visibility and trust within the team on what is being tested during every software delivery cycle.

Integration of Galen in development process

TDD

  • For every new page layout story a “3 Amigos” task is defined in the sprint planning meeting.
  • “3 Amigos” task is considered to be the first task for every story in the sprint.
  • Immediately after the planning the Product Owner, tester, and a developer create a meeting and collaborate together to define what they want to test.
  • The tests are written as Galen spec by a pair of developers or by a developer-tester pair.
  • Development of the code is started in parallel to, or after, the implementation of the tests. These then work as executable acceptance test.
comments powered by Disqus