Category Archives: Make your Tests Maintainable

A guide to clear assertions with Hamcrest

A good practice in test automation is the use of Descriptive And Meaningful Phrases (also known as DAMP). This means that our tests clearly tell us what they do in language that is relevant to the domain. My goal for this article is to guide you through the steps to making descriptive and meaningful assertions with Hamcrest. I will do so by tackling the following issues.

  1. Readability of our assertions
  2. Completeness of the output of our assertions
  3. Duplication of assertion fail information in our tests

Let’s assume we have a class SpeedCamera with a method isPictureTaken which returns a boolean. We’ll skip the setup of the test for brevity’s sake. Let’s write the following JUnit assertion.


public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertFalse(speedCamera.isPictureTaken()); 
} 
  // failure output: java.lang.AssertionError 
  // org.junit.Assert.fail(Assert.java:86) .....long stack trace
1
2
3
4
5
6
public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertFalse(speedCamera.isPictureTaken()); 
} 
  // failure output: java.lang.AssertionError 
  // org.junit.Assert.fail(Assert.java:86) .....long stack trace

Readability
Note that the assertion is not very readable and its failure output is incomplete. We’ll take our first step in solving the readability issue with Hamcrest.


public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertThat(speedCamera.isPictureTaken(), equalTo(false)); 
} 
  /* failure output: java.lang.AssertionError: 
     Expected: <false>
     but: was <true>*/
1
2
3
4
5
6
7
public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertThat(speedCamera.isPictureTaken(), equalTo(false)); 
} 
  /* failure output: java.lang.AssertionError: 
     Expected: <false>
     but: was <true>*/

Completeness
Our test reads like a sentence, and even our failure output has improved a bit. But we miss important failure information for our test report. If we were to read the test report, for instance after running a test suite, we would not know which method expected false but was true without having to go back to the test code. We can solve this isue by adding a description to our test like this.


public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53);
  assertThat("isPictureTaken should have returned false",
              speedCamera.isPictureTaken(), equalTo(false));
} 
  /*failure output: java.lang.AssertionError: isPictureTaken should have returned false 
    Expected: <false> 
    but: was <true>*/ 
1
2
3
4
5
6
7
8
public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53);
  assertThat("isPictureTaken should have returned false",
              speedCamera.isPictureTaken(), equalTo(false));
} 
  /*failure output: java.lang.AssertionError: isPictureTaken should have returned false 
    Expected: <false> 
    but: was <true>*/ 

Duplication
Our test failure output now tells us which method returns the wrong value. But if we continue to use this approach, we create the problem of duplication of our test failure information as we can see in this example.


public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53);
  assertThat("isPictureTaken should have returned false", 
             speedCamera.isPictureTaken(), equalTo(false));
}

public void shouldTakePicture() {
  vehicle.passSpeedCamera(54);
  assertThat("isPictureTaken should have returned true",
             speedCamera.isPictureTaken(), equalTo(true)); 
}

// ad infinitum...
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53);
  assertThat("isPictureTaken should have returned false", 
             speedCamera.isPictureTaken(), equalTo(false));
}

public void shouldTakePicture() {
  vehicle.passSpeedCamera(54);
  assertThat("isPictureTaken should have returned true",
             speedCamera.isPictureTaken(), equalTo(true)); 
}

// ad infinitum...

But we can solve the duplication issue by implementing customised BaseMatchers. Let’s assume our SpeedCamera has three methods getCorrectedSpeed, isPictureTaken and isLicenseRevoked. We’ll use the TypeSafeMatcher because it’s straightforward in usage. Check out the Hamcrest API for more options.


public class SpeedCameraMatchers {

  public static BaseMatcher<SpeedCamera> hasCorrectedSpeedTo(final int correctedSpeed) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("The camera should have corrected the speed to ")
                   .appendValue(correctedSpeed);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" alas, the camera corrected it to ")
                           .appendValue(camera.getCorrectedSpeed());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return correctedSpeed == camera.getCorrectedSpeed();
      }
    };
  }

  public static BaseMatcher<SpeedCamera> hasTakenAPicture(final boolean hasTakenPicture) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("Taking a picture should have returned ")
                   .appendValue(hasTakenPicture);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" returned ")
                           .appendValue(camera.isPictureTaken());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return hasTakenPicture == camera.isPictureTaken();
      }
    };
  }

  public static BaseMatcher<SpeedCamera> hasRevokedLicense(final boolean hasRevokedLicense) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("Revoking license should have returned ")
                   .appendValue(hasRevokedLicense);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" returned ")
                           .appendValue(camera.isLicenseRevoked());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return hasRevokedLicense == camera.isLicenseRevoked();
      }
    };
  } 
} 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
public class SpeedCameraMatchers {

  public static BaseMatcher<SpeedCamera> hasCorrectedSpeedTo(final int correctedSpeed) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("The camera should have corrected the speed to ")
                   .appendValue(correctedSpeed);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" alas, the camera corrected it to ")
                           .appendValue(camera.getCorrectedSpeed());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return correctedSpeed == camera.getCorrectedSpeed();
      }
    };
  }

  public static BaseMatcher<SpeedCamera> hasTakenAPicture(final boolean hasTakenPicture) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("Taking a picture should have returned ")
                   .appendValue(hasTakenPicture);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" returned ")
                           .appendValue(camera.isPictureTaken());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return hasTakenPicture == camera.isPictureTaken();
      }
    };
  }

  public static BaseMatcher<SpeedCamera> hasRevokedLicense(final boolean hasRevokedLicense) { 
    return new TypeSafeMatcher<SpeedCamera>() {

      public void describeTo(final Description description) {
        description.appendText("Revoking license should have returned ")
                   .appendValue(hasRevokedLicense);
      }
    
      public void describeMismatchSafely(final SpeedCamera camera, final Description mismatchDescription) {
        mismatchDescription.appendText(" returned ")
                           .appendValue(camera.isLicenseRevoked());
      }
    
      public boolean matchesSafely(final SpeedCamera camera) {
        return hasRevokedLicense == camera.isLicenseRevoked();
      }
    };
  } 
} 

Now our assertions are even more readable, their output provides us with complete information and we’re able to reuse them without duplicating code.


public void shouldCorrectSpeedToZero() { 
  vehicle.passSpeedCamera(0); 
  assertThat(speedCamera, hasCorrectedSpeedTo(0)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: The camera should have corrected the speed to <0> 
  but: alas, the camera corrected it to <-3>*/

public void shouldCorrectSpeedToHundred() { 
  vehicle.passSpeedCamera(105); 
  assertThat(speedCamera, hasCorrectedSpeedTo(100)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: The camera should have corrected the speed to <100> 
  but: alas, the camera corrected it to <99>*/

public void shouldTakePicture() { 
  vehicle.passSpeedCamera(54); 
  assertThat(speedCamera, hasTakenAPicture(true)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Taking a picture should have returned <true> 
  but: returned <false>*/

public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertThat(speedCamera, hasTakenAPicture(false)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Taking a picture should have returned <false> 
  but: returned <true>*/

public void shouldRevokeLicense() { 
  vehicle.passSpeedCamera(105); 
  assertThat(speedCamera, hasRevokedLicense(true)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Revoking license should have returned <true>
  but: returned <false>*/

public void shouldNotRevokeLicense() { 
  vehicle.passSpeedCamera(104);
  assertThat(speedCamera, hasRevokedLicense(false)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: Revoking license should have returned <false>
  but: returned <true>*/ 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
public void shouldCorrectSpeedToZero() { 
  vehicle.passSpeedCamera(0); 
  assertThat(speedCamera, hasCorrectedSpeedTo(0)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: The camera should have corrected the speed to <0> 
  but: alas, the camera corrected it to <-3>*/

public void shouldCorrectSpeedToHundred() { 
  vehicle.passSpeedCamera(105); 
  assertThat(speedCamera, hasCorrectedSpeedTo(100)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: The camera should have corrected the speed to <100> 
  but: alas, the camera corrected it to <99>*/

public void shouldTakePicture() { 
  vehicle.passSpeedCamera(54); 
  assertThat(speedCamera, hasTakenAPicture(true)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Taking a picture should have returned <true> 
  but: returned <false>*/

public void shouldNotTakePicture() { 
  vehicle.passSpeedCamera(53); 
  assertThat(speedCamera, hasTakenAPicture(false)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Taking a picture should have returned <false> 
  but: returned <true>*/

public void shouldRevokeLicense() { 
  vehicle.passSpeedCamera(105); 
  assertThat(speedCamera, hasRevokedLicense(true)); 
} 
/*failure output: java.lang.AssertionError:
  Expected: Revoking license should have returned <true>
  but: returned <false>*/

public void shouldNotRevokeLicense() { 
  vehicle.passSpeedCamera(104);
  assertThat(speedCamera, hasRevokedLicense(false)); 
} 
/*failure output: java.lang.AssertionError: 
  Expected: Revoking license should have returned <false>
  but: returned <true>*/ 

In my next article we will look at a safe way to chain our freshly made custom BaseMatchers.

Page Object Model – Introduction

Creating Selenium test cases can result in an unmaintainable project. One of the reasons is that too many duplicated code is used. Duplicated code could be caused by duplicated functionality and this will result in duplicated usage of locators. The disadvantage of duplicated code is that the project is less maintainable. If some locator will change, you have to walk through the whole test code to adjust locators where necessary. By using the page object model we can make non-brittle test code and reduce or eliminate duplicate test code. Beside of that it improves the readability and allow us to create interactive documentation. Last but not least, we can create tests with less keystrokes. An implementation of the page object model can be achieved by separating the abstraction of the test object and the test scripts.

This section of the blog will explain the common use of the page object model. We will make use of an Integrate Development Environment, called Eclipse.

Set up a Selenium project environment

Problem

This recipe will describe how we can set up our project environment. We will use Eclipse as development environment as further examples will be written in Java, but we can use any IDE which support our favorite programming language, preferable a strong type language. (like: ActionScript 3, C++, C#, Java, Python, OCaml, Vala) A strong type language sets the boundaries where we have to behave in, other than a dynamic type language(like: BASIC, JavaScript, Perl, PHP, Rexx).

Prerequisites

Make sure you have download the latest stable TestNG library from http://www.testng.org. Next thing is to make sure that we have the latest stable version of the Selenium library from http://code.google.com/p/selenium/downloads/list.

Solution

  1. Create a new Java project in Eclipse
  2. Create two separate folders, named tests and src
  3. Import the TestNG library. By opening the context menu on the project, and select Properties > Java Build Path > Libraries > Add External JARs
  4. Import the Selenium library in the same way as described above

What has been done

We have create our project structure, by importing the libraries and creating two separate folders. In the src folder we will create the website abstraction and in the tests folder we will create our test scripts.

Tip 1: The latest Selenium releases comes with a lot of additional libraries which are very useful and can be found in the libs folder.

Model the application interface

The first step we have to take in implementing the page object model is that we have to model the user experience. This means that all page specific elements has to be extracted to separate classes. This will guarantee that all functionality will be scripted only once.

Getting ready
The tests drive the implementation of the page object. In this way, we never end up with unused page object code.

How to do it…

  1. Create a new class file and refer the name to the actual page from the test object, by opening the context menu on the src folder and select New > Class. Make sure you fill in a package name and class name. Package name should refer to the entire website and the class name should refer to the specific page.
  2. Import the Selenium package in order to use the functions in the API. The code will look like this:
package prestashop;

import org.openqa.selenium.*;

public class SearchPage {

}

Expose methods

We can expose methods in order to reduce duplicated code. We are able to call the method multiple times. This will ensure a better maintainable test code, because we only have to make adjustments and improvements in one particular place.

**Getting ready
**Search for duplicated functionality we use in our tests. For example the ‘login’ functionality. The selenium actions we have to provide to login will look like this:

WebElement emailEl = driver.findElement(By.name("email"));
emailEl.sendKeys("test@test.com");
WebElement passwordEl = driver.findElement(By.name("password"));
passwordEl.sendKeys("1qazxsw2");
WebElement loginForm = driver.findElement(By.name("login"));
loginForm.submit();

 

How to do it…
We can simple wrap the described functionality in a method and we can give it a sensible name. We can create the method like this:

public void login() {
        WebElement emailEl = driver.findElement(By.name("email"));
        emailEl.sendKeys("test@test.com");
        WebElement passwordEl = driver.findElement(By.name("password"));
        passwordEl.sendKeys("1qazxsw2");
        WebElement loginForm = driver.findElement(By.name("login"));
        loginForm.submit();
    }

We can call this function using the following code:

login();

How it works…
We can simple call the method from our test script, once we have to add one item to the cart. The sensible method name tells use directly what the selenium API calls are doing.

There’s more…

Passing parameters through methods
We can pass parameters through methods, just as in normal programming code. The code below will show us how we can login with parameterized email and password.

public void login(String email, String password) {
        WebElement emailEl = driver.findElement(By.name("email"));
        emailEl.sendKeys(email);
        WebElement passwordEl = driver.findElement(By.name("password"));
        passwordEl.sendKeys(password);
        WebElement loginForm = driver.findElement(By.name("login"));
        loginForm.submit();
    }

We can call this function using the following code:

login("test@test.com", "1qazxsw2");

Dealing with moving focus

Problem

Imagine the following situation: after accepting a confirmation dialog you will be redirected to another page. This recipe will explain how to deal with this inevitable situation.

Prerequisites

We have made a class file for every unique page. So in theory every page is accessible.

How to do it…

We have to change the return-type of the method, in this case we set it to MemberPage. The doLogin method will look like this:


public MemberPage doLogin(String email, String password) {
    WebElement emailEl = driver.findElement(By.name("email"));
    emailEl.sendKeys(email);
    WebElement passwordEl = driver.findElement(By.name("password"));
    passwordEl.sendKeys(password);
    WebElement loginForm = driver.findElement(By.name("login"));
    loginForm.submit();
    return new MemberPage();
}
1
2
3
4
5
6
7
8
9
public MemberPage doLogin(String email, String password) {
    WebElement emailEl = driver.findElement(By.name("email"));
    emailEl.sendKeys(email);
    WebElement passwordEl = driver.findElement(By.name("password"));
    passwordEl.sendKeys(password);
    WebElement loginForm = driver.findElement(By.name("login"));
    loginForm.submit();
    return new MemberPage();
}

How it works…

The MemberPage object is returned, once we call the doLogin method. From our testscript perspective we can access all the public methods in the MemberPage class.

Ambiguous focus

Set the return-type to void, if the focus of an certain action is ambiguous. Like in highly dynamic websites, where we have no control on the input/output data. In this case we can use the code below:


public void doLogin(String email, String password) {
    WebElement emailEl = driver.findElement(By.name("email"));
    emailEl.sendKeys(email);
    WebElement passwordEl = driver.findElement(By.name("password"));
    passwordEl.sendKeys(password);
    WebElement loginForm = driver.findElement(By.name("login"));
    loginForm.submit();
}
1
2
3
4
5
6
7
8
public void doLogin(String email, String password) {
    WebElement emailEl = driver.findElement(By.name("email"));
    emailEl.sendKeys(email);
    WebElement passwordEl = driver.findElement(By.name("password"));
    passwordEl.sendKeys(password);
    WebElement loginForm = driver.findElement(By.name("login"));
    loginForm.submit();
}