Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Year: 2015

Getting Selenium Test Running On Gradle Framework

Gradle + Selenium is probably not a popular framework yet when I was developing it as I find it difficult to look for relevant resources online. There were bits and pieces of information here and there which I then glue then up together.

My build.gradle contains the following implementations:

(1) Add cucumberRuntime under Configurations

configurations {
cucumberRuntime {
extendsFrom testRuntime
}

}

(2) Source sets

sourceSets{
test{
groovy{
srcDirs ‘src/test’
}
}

}

(3) Compile test output

task testJar(type: Jar){
from(sourceSets.test.output){
include ‘**/*.class’
exclude ‘**/*StepDef*.class’
}

}

(4) Glue everything together

task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {
            main = "cucumber.api.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['-f', 'json:target/cucumber-report.json','--plugin','pretty', '--glue', 'src/test/groovy/au/com/story/steps', 'src/test/groovy/au/com/story/features']
        }
    }
}

Some Workarounds In PACT Test

Writing PACT test might be quite a challenge in the beginning, as it is a new tool, and we might not be able to find much information in the web yet.

Personally, I find the PACT Google Support Group is very helpful when I encounter issues. Otherwise, sometimes I do some trial and error to solve my obstacles. Below are some quick solutions to some consumer/provider test issues that I have encountered.

Consumer Tests

Below are some scenarios when it comes to writing Body DSL for some JSON bodies.

Time stamp with certain format

Eg:  2015-05-13T06:18:06Z

Body DSL: yyyy-MM-dd’T’HH:mm:ssX

Negative integer

Eg: negative integer: -280

Body DSL: integer(-250)

Field with empty []

Eg: factory[]

Body DSL: factory([])

JSON body inside arrays

Eg: factory[{

field1 123

field2 12.5

}]

Body DSL:

factory([{

field1 integer()

field2 real()

}])

A field which might return an empty value

Eg:

“fruit”, a String which can be an empty String sometimes

Body DSL:

“fruit” (~/\w.+|^$/,”banana or apple”)

Note that we could use an or operator inside the regex. However, it is probably better to separate this into 2 test cases.

Provider Test

Unable to call API in Provider test via load balancer

Eg:

API connection fails when we call an API via a URL that goes through load balancer (not sure why)

Workaround:

Call the API directly via the server, bypassing load balancer

Building A Selenium Web Driver Test Framework From Scratch

Selenium UI test is the most used high level test these days for web applications. Regardless of which language you use, the framework is pretty much the same. I would like to share with everybody the Selenium web driver testing framework I worked on recently using Selenium in Java on a Groovy Gradle framework.

I will provide the step-by-step guideline on how to create a skeleton framework from scratch:

  1. Import libraries you need

dependencies {

    compile 'org.codehaus.groovy:groovy-all:2.4.3'
    testCompile 'junit:junit:4.12'
    testCompile 'org.seleniumhq.selenium:selenium-java:2.46.0'
    testCompile 'info.cukes:cucumber-groovy:1.2.3'
}

You will need the Selenium Java from Maven repository, Cucumber in Groovy as a BDD layer (although this is not necessary), JUnit to assert test values.

2. Create a webdriver class

import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement
import org.openqa.selenium.chrome.ChromeDriver

public class CsWebDriver {

 private static WebDriver _driver

 public static WebDriver get(String browserType)
 {
 if (browserType.equals("chrome"))
 {
 System.setProperty("webdriver.chrome.driver","srcmainresourceschromedriver.exe")
 ChromeOptions options = new ChromeOptions();
 options.setBinary("c:Chrome.V41GoogleChromePortable.exe");

 _driver=new ChromeDriver(options)

 }

 return _driver

 }

This is where we declare a new web driver object from the library and initialize it with a copy of driver downloaded from  ChromeDriver. You could create or extend all browser related functionalities in this class, such as Open a URL, Quit, Wait, Move To Click, etc.

For example: Move to an element to click using Action class

public static WebElement WaitTillElementExist(WebElement ele, int seconds)
{

   WebDriverWait wait = new WebDriverWait(_driver,seconds)
   return wait.until(ExpectedConditions.elementToBeClickable(ele))
}

3. Create Page Factory to host Page Objects

public class PageFactory {

    private static WebDriver _driver

    public static LoginPage LoginPage

   public static void InitialisePages(WebDriver driver)
   {
     _driver=driver
     LoginPage= new LoginPage(driver)
   }
   public static void ResetPages()
   {
     LoginPage=null
   }

 public static GetDriver() 
 { return _driver 
 }

Page Factory above is where we declare, initialize, and reset our Page Objects. An instance of web driver will be passed into each Page Object.

import org.openqa.selenium.support.FindBy 
import org.openqa.selenium.support.How

@InheritConstructors
class LoginPage extends BasePage {

    @FindBy(how=How.ID, using="Username")
    private WebElement userName

    @FindBy(how=How.ID, using="Password")
    private WebElement password
    @FindBy(how=How.CSS, using="button[class='btn']")
    private WebElement loginBtn

    public void Login(String username, String pwd)
    {
      userName.sendKeys(username)
      password.sendKeys(pwd)
      loginBtn.click()
    }
}

Every webpage that we are testing is treated as an object. By importing the Selenium library FindBy and How we can select the element that we want using ID, CSS, XPATH, etc. The page object will also contain functions specific to the page, for eg. Login function. As we can see, the page object is extended from the BasePage. A BasePage looks like this:

import org.openqa.selenium.support.PageFactory

public class BasePage {

    public WebDriver driver

    public BasePage(WebDriver webdriver)
    {
        this.driver=webdriver
        PageFactory.initElements(webdriver,this)

    }
}

We use the Selenium PageFactory library to easily implement this.

    4. Binding everything together

So we now have the web driver class, page factory, and page objects, how do we glue this all together into an executable test? To make test easier to understand and read, we usually like to use a BDD tool to describe a test Scenario. I will not go into depth on the BDD tool in this post, but I would like to touch base on the Before Scenario Hook of a BDD tool that is used to glue all pieces together.

this.metaClass.mixin(cucumber.api.groovy.Hooks) this.metaClass.mixin(cucumber.api.groovy.EN) Before("@web") 
{ 
  scenario -> PageFactory.InitialisePages(WebDriver.get("chrome")) 
  WebDriver.Open(oagUrl) 
}

Before every scenario, we initialize all Page Objects with a Chrome driver and then  open the URL.

After("@web") { scenario ->

    if (scenario.isFailed()) {
        String fileName=scenario.getName()
        WebDriver.PrintPageSource()
        WebDriver.TakeScreenShot(fileName, scenario)
    }

    WebDriver.Quit()
    PageFactory.ResetPages()
}

After every scenario, we check if the scenario is failed, if so, we print out the scenario name and take a screen shot. We then quit the driver and reset all Page Objects to null.

  5. Run the test via command line

To enable the test to be run on a CI box, we have to make the test run via command line. The example below is using Gradle to run the Cucumber Groovy test:

configurations {
    cucumberRuntime {
        extendsFrom testRuntime
    }
}
sourceSets{
    test{
        groovy{
            srcDirs 'src/test'
        }
    }
}
task cucumber() {
    dependsOn assemble, compileTestJava
    doLast {
        javaexec {
            main = "cucumber.api.cli.Main"
            classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
            args = ['-f', 'json:target/cucumber-report.json','--plugin','pretty', '--glue', 'src/test/groovy/au/com/test/story/steps', 'src/test/groovy/au/com/test/story/features']
        }
    }
}

We created a Gradle task called “cucumber” and execute the test using “javaexec”.  We need to define in the Source Sets the directory of the test. In “args –glue”, we define where the Feature  files and Steps Definitions are.

  6. Finally, run your test!

PACT: How To Write Provider Test

Writing PACT provider test is easy because you do not need to write them, you only need to call them!

Example below is in Groovy using the Gradle framework.

(1) Import the library and include it in the build script

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath ‘au.com.dius:pact-jvm-provider-gradle_2.11:3.0.1’
    }
}

(2) In build.gradle, call the API, which is the Provider and define the location of the PACT JSON file generated by the Consumer test. These are done under a “pact” segment.

pact {

    dataProviders {

         DataProvider {
            protocol = ‘http’
            host = ‘test.api.com.au’
            port = 80
            path = ‘/consumer/search’

             hasPactWith(‘Consumer’) {
                 pactFile = file(‘..\ApiTests\target\pacts\Consumer-Provider.json’)

            }

        }

}
 
The API that we want to call is http://test.api.com.au:80/consumer/search. Consumer-Provider.json is the PACT file generated by the Consumer test in my previous Consumer test blog. The file name “Consumer-Provider.json” is based on the “serviceConsumer” and “hasPactWith” definitions in the Consumer test. “APITests” is the root directory of the project, while “target” is the default directory where PACT files are going to be generated. However, this maybe overwritten (refer to the original GitHub wiki on how to do this). “hasPactWith(‘Consumer’)” in Provider test is merely a name. This is useful so we can run each test individually.
You may define as many tests as you want inside the “pact” segment.

    @Test
void “Consumer test for Consumer and Provider”() {
def consumer_service = new PactBuilder()
consumer_service {
serviceConsumer “Consumer”
hasPactWith “Provider”



(3) Execute the command gradle pactVerify from the root directory of the project. Or to call the test individually, we may specify gradle pactVerify -Ppact.filter.consumer=[hasPactWith value]Example of output:

PACT Provider will call the API that you defined in gradle.build and compare the actual output against the output generated by Consumer test. It compares for the response status, content type, and body as how you defined them.

This way, we know that the API contract that the Consumer expects is similar as what the API Provider produces.

Example of a PACT Provider failure:

 

In the example above, value Type is returned as “Car” but it is defined as an integer in the Consumer test.

PACT: How To Write Consumer Test

Below is an example of a PACT Consumer tests with example in Groovy.

  1. Import the library
Ensure that we downloaded the PACT consumer library from Maven repository import them in your consumer test file. We also need to include REST Client to make API calls and JUnit for assertion.
import au.com.dius.pact.consumer.PactVerified$
import au.com.dius.pact.consumer.groovy.PactBuilder
import groovyx.net.http.RESTClient
import org.junit.Test

    2. Define the consumer service that will consume the API provider


Example below is a consumer test class that defines a consumer service “Consumer” that is requesting data from API provider “Provider”. The API request is a GET method with parameter “q” with some value. Upon success, it returns a response code of 200 with Content-Type of “application/json;charset=UTF-8”. 

class ConsumerTests {

    @Test
    void “Consumer test for Consumer and Provider”() {
        def consumer_service = new PactBuilder()
        consumer_service {
            serviceConsumer “Consumer”
            hasPactWith “Provider”
            port 1237

            given(‘Scenario of Consumer is retrieving data from Provider’)
            uponReceiving(‘a retrieve a data request’)
            withAttributes(method: ‘get’, path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])
            willRespondWith(
                    status: 200,
                    headers: [‘Content-Type’: ‘application/json; charset=UTF-8’]
            )

            withBody {

 }

3. Define the response body

There are 2 ways to define the response body.

(a) Response body with exact or fixed data.

Use this method if you need to test that the data returned by the API is exactly the same as what you defined in the consumer test.

For example:

  withBody{
                “Id” (“87992”)
                “Seller” (“Michael”)
                “CompanyName” (“ABC123”)
}

The example above will generate a PACT JSON file that looks like:

  “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “Seller” : “Michael”,
        “CompanyName” : “ABC123”
      }
    }

So the PACT provider test will check for a response with body with exactly the same structure and data as the PACT JSON file.

(b) Response body with flexible data

Use this method if you only want to test that the contract or structure of the API remains the same. This is useful in the scenario when data changes frequently and the effort of test data maintenance is high.

For example:

  withBody{
                “Id” regexp(~/w.+/,”87992″)
                “SellerGuid” regexp(~/w.+/,”Michael”)
                “CompanyName” regexp(~/w.+/, “ABC123”)
}

The example above will generate PACT JSON file that looks like:

 “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “SellerGuid” : “Michael”,
        “CompanyName” : “ABC123”
      },
      “matchingRules” : {
        “$.body.Id” : {
          “regex” : “\w.+”
        },
        “$.body.SellerGuid” : {
          “regex” : “\w.+”
        },
        “$.body.CompanyName” : {
          “regex” : “\w.+”
        }
      }

Instead of passing the exact data to match, we pass in a regular expression instead. Therefore, a set of matching rules will be generated in the PACT JSON file. PACT Provider test will look at the matching rules and ignore the data. So, if Company Name is changed from “ABC123” to “ABC678”, the test will still pass as the data still matches the regular expression defined.

This is a good example of a consumer with body that uses flexible matching of many different data types such as string, integer, IP, time stamp, etc.

Another key features of the flexible matching is the flexible array matching.

For example:

    hits minLike(1) {
                        _index(~/w.+/, “12345”)
                        _type(~/w.+/, “Dog”)
                        _id(~/d+/, “263287”)
                        _score real()

The above is useful if the JSON output that you want to test has array structures and the number of arrays in the structure changes.

More information can be found here

(4) Generate the PACT file

The last part of the consumer test is basically generating the PACT JSON output.


For example:

   def result = consumer_service.run() {
                def client = new RESTClient(‘http://localhost:1237/’)

                def response = client.get(path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])

                assert response.status == 200
   
                consumer_service.buildInteractions()
            }


If everything is defined correctly, a JSON output will be generated in the file structure under the “target” directory.

The JSON output is contract of the API that you have defined in the consumer test, which is the API that the consumer consumes.

Introduction: API Contract Testing With PACT

PACT is a tool introduced to enable to testing of API contract between the consumer (the system that uses the API) and the provider (the system that produces the API). It is available in a couple of different languages. The version that I have been using is the JVM version in Groovy. This tool is very useful to detect API structure changes. It is good in the sense that we can test the structure  of the API irrespective of the data (we can just test the data type). It could be easily integrated into CI box as well.

 

Consumer Test

Consumer test defines the contract or structure of the API that the system uses. It can be used to enable TDD and in a situation when the system that consumes the API is built before the API exist. Upon successful consumer test run, a JSON file that mimic the actual API output will be generated under the “target” folder.

Provider Test

Provider test basically is a call to the actual API which then compares the actual API result with the structure which we expect (as defined in the Consumer test). It is built in the build script depending on which framework that we use, in my case it is called in the build.grade file.
I will include more specific examples with descriptions on the Consumer and Provider tests in my upcoming posts.

Using JavaScript To Manipulate The Web Page

There are circumstances when we need to manipulate the web page in order to test some scenarios:
 
Below are some examples in C#:
 

Use JavaScript to Change Value

 
Changing value of an input field with ID input-confirm to iambot to simulate SpamBot’s action:
IJavaScriptExecutor js = driver as IJavaScriptExecutor;
js.ExecuteScript(“$(‘#input-confirm’).val(‘iamabot’)”);
 
 

Use JavaScript to Disable Client Side Validation 

 
 
Remove JavaScript validation classes from submit button and form before clicking on button. This test case is to ensure that we do have server side validation in place.
 

var js = driver as IJavaScriptExecutor;

js.ExecuteScript(“$(‘button.primary-cta.button.fn_validate_submit’).removeClass(‘button.primary-cta.button.fn_validate_submit disabled’)”);

js.ExecuteScript(“$(‘form’).removeClass(‘form fn_validate fn_validate_container’)”);

Submit.Click();

Happy 2015!

My last blog stops during December 2014, after the MeetUp @ Carsales. I mentioned that I try to blog once a month, but I have not been able to do that this year.
2015 is a beginning of a great year. I resigned from my position at PageUp at the end of January 2015. Then I was away for holiday during the entire month of February. In Chinese sayings, “taking a break is to get prepared for a longer journey”. I had quite a bit of adventures during the break, including my first roller coaster ride @ Legoland Malaysia.

 

At one point, I reached to a point where I wonder where I want to be in terms of career, continue in the technical path or more towards leadership. I guess job satisfaction and self happiness matters the most to me. I have always been passionate to learn about the latest technology and best testing practices. Learning, learning, and keep learning is what that could drive me forward. And I love hands on job!
Therefore, I have decided to choose the technical path. I commenced my new role as Senior Automation Test Engineer at UniSuper right after my break in Malaysia.
My Testing Resolutions for 2015 are:
  • Do as much hands on job as possible!
  • Do better in every thing that I have already  knew (not just learning new stuff). For e.g. use more CSS selectors compared to XPath as it is faster. 🙂
  • Meet more and learn more from knowledgeable peers and colleagues
  • Continue learning…and learning……!
  • More balanced work & life……more exercise and charity work 🙂
I have also been doing testing for over 11 years. Throughout the journey, I considered myself as the lucky girl as I have met many good and helpful bosses and colleagues since my first job. I am thankful to them in many ways.
  • Kalai Murugiah, my direct manager from BMM who has given me lots of learning opportunity. Anna Fernando, the general manager who has made the 21 year old girl feel confident and comfortable at her first job.
  • Malcolm Webster, my manager at REA who has been a great leader helping me grow in many aspects
  • Lots of great colleagues and peers whom I keep in touch till today…….where we share our ups and downs of work and also support during tough times (Andri).
I had a great journey….2014 marked the end of an era, and 2015 is the beginning of the next! 🙂

 

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑