Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Tag: API

Some Workarounds In PACT Test

Writing PACT test might be quite a challenge in the beginning, as it is a new tool, and we might not be able to find much information in the web yet.

Personally, I find the PACT Google Support Group is very helpful when I encounter issues. Otherwise, sometimes I do some trial and error to solve my obstacles. Below are some quick solutions to some consumer/provider test issues that I have encountered.

Consumer Tests

Below are some scenarios when it comes to writing Body DSL for some JSON bodies.

Time stamp with certain format

Eg:  2015-05-13T06:18:06Z

Body DSL: yyyy-MM-dd’T’HH:mm:ssX

Negative integer

Eg: negative integer: -280

Body DSL: integer(-250)

Field with empty []

Eg: factory[]

Body DSL: factory([])

JSON body inside arrays

Eg: factory[{

field1 123

field2 12.5

}]

Body DSL:

factory([{

field1 integer()

field2 real()

}])

A field which might return an empty value

Eg:

“fruit”, a String which can be an empty String sometimes

Body DSL:

“fruit” (~/\w.+|^$/,”banana or apple”)

Note that we could use an or operator inside the regex. However, it is probably better to separate this into 2 test cases.

Provider Test

Unable to call API in Provider test via load balancer

Eg:

API connection fails when we call an API via a URL that goes through load balancer (not sure why)

Workaround:

Call the API directly via the server, bypassing load balancer

PACT: How To Write Provider Test

Writing PACT provider test is easy because you do not need to write them, you only need to call them!

Example below is in Groovy using the Gradle framework.

(1) Import the library and include it in the build script

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath ‘au.com.dius:pact-jvm-provider-gradle_2.11:3.0.1’
    }
}

(2) In build.gradle, call the API, which is the Provider and define the location of the PACT JSON file generated by the Consumer test. These are done under a “pact” segment.

pact {

    dataProviders {

         DataProvider {
            protocol = ‘http’
            host = ‘test.api.com.au’
            port = 80
            path = ‘/consumer/search’

             hasPactWith(‘Consumer’) {
                 pactFile = file(‘..\ApiTests\target\pacts\Consumer-Provider.json’)

            }

        }

}
 
The API that we want to call is http://test.api.com.au:80/consumer/search. Consumer-Provider.json is the PACT file generated by the Consumer test in my previous Consumer test blog. The file name “Consumer-Provider.json” is based on the “serviceConsumer” and “hasPactWith” definitions in the Consumer test. “APITests” is the root directory of the project, while “target” is the default directory where PACT files are going to be generated. However, this maybe overwritten (refer to the original GitHub wiki on how to do this). “hasPactWith(‘Consumer’)” in Provider test is merely a name. This is useful so we can run each test individually.
You may define as many tests as you want inside the “pact” segment.

    @Test
void “Consumer test for Consumer and Provider”() {
def consumer_service = new PactBuilder()
consumer_service {
serviceConsumer “Consumer”
hasPactWith “Provider”



(3) Execute the command gradle pactVerify from the root directory of the project. Or to call the test individually, we may specify gradle pactVerify -Ppact.filter.consumer=[hasPactWith value]Example of output:

PACT Provider will call the API that you defined in gradle.build and compare the actual output against the output generated by Consumer test. It compares for the response status, content type, and body as how you defined them.

This way, we know that the API contract that the Consumer expects is similar as what the API Provider produces.

Example of a PACT Provider failure:

 

In the example above, value Type is returned as “Car” but it is defined as an integer in the Consumer test.

PACT: How To Write Consumer Test

Below is an example of a PACT Consumer tests with example in Groovy.

  1. Import the library
Ensure that we downloaded the PACT consumer library from Maven repository import them in your consumer test file. We also need to include REST Client to make API calls and JUnit for assertion.
import au.com.dius.pact.consumer.PactVerified$
import au.com.dius.pact.consumer.groovy.PactBuilder
import groovyx.net.http.RESTClient
import org.junit.Test

    2. Define the consumer service that will consume the API provider


Example below is a consumer test class that defines a consumer service “Consumer” that is requesting data from API provider “Provider”. The API request is a GET method with parameter “q” with some value. Upon success, it returns a response code of 200 with Content-Type of “application/json;charset=UTF-8”. 

class ConsumerTests {

    @Test
    void “Consumer test for Consumer and Provider”() {
        def consumer_service = new PactBuilder()
        consumer_service {
            serviceConsumer “Consumer”
            hasPactWith “Provider”
            port 1237

            given(‘Scenario of Consumer is retrieving data from Provider’)
            uponReceiving(‘a retrieve a data request’)
            withAttributes(method: ‘get’, path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])
            willRespondWith(
                    status: 200,
                    headers: [‘Content-Type’: ‘application/json; charset=UTF-8’]
            )

            withBody {

 }

3. Define the response body

There are 2 ways to define the response body.

(a) Response body with exact or fixed data.

Use this method if you need to test that the data returned by the API is exactly the same as what you defined in the consumer test.

For example:

  withBody{
                “Id” (“87992”)
                “Seller” (“Michael”)
                “CompanyName” (“ABC123”)
}

The example above will generate a PACT JSON file that looks like:

  “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “Seller” : “Michael”,
        “CompanyName” : “ABC123”
      }
    }

So the PACT provider test will check for a response with body with exactly the same structure and data as the PACT JSON file.

(b) Response body with flexible data

Use this method if you only want to test that the contract or structure of the API remains the same. This is useful in the scenario when data changes frequently and the effort of test data maintenance is high.

For example:

  withBody{
                “Id” regexp(~/w.+/,”87992″)
                “SellerGuid” regexp(~/w.+/,”Michael”)
                “CompanyName” regexp(~/w.+/, “ABC123”)
}

The example above will generate PACT JSON file that looks like:

 “response” : {
      “status” : 200,
      “headers” : {
        “Content-Type” : “application/json;charset=utf-8”
      },
      “body” : {
        “Id” : “87992”,
        “SellerGuid” : “Michael”,
        “CompanyName” : “ABC123”
      },
      “matchingRules” : {
        “$.body.Id” : {
          “regex” : “\w.+”
        },
        “$.body.SellerGuid” : {
          “regex” : “\w.+”
        },
        “$.body.CompanyName” : {
          “regex” : “\w.+”
        }
      }

Instead of passing the exact data to match, we pass in a regular expression instead. Therefore, a set of matching rules will be generated in the PACT JSON file. PACT Provider test will look at the matching rules and ignore the data. So, if Company Name is changed from “ABC123” to “ABC678”, the test will still pass as the data still matches the regular expression defined.

This is a good example of a consumer with body that uses flexible matching of many different data types such as string, integer, IP, time stamp, etc.

Another key features of the flexible matching is the flexible array matching.

For example:

    hits minLike(1) {
                        _index(~/w.+/, “12345”)
                        _type(~/w.+/, “Dog”)
                        _id(~/d+/, “263287”)
                        _score real()

The above is useful if the JSON output that you want to test has array structures and the number of arrays in the structure changes.

More information can be found here

(4) Generate the PACT file

The last part of the consumer test is basically generating the PACT JSON output.


For example:

   def result = consumer_service.run() {
                def client = new RESTClient(‘http://localhost:1237/’)

                def response = client.get(path: ‘/consumer/search’, query: [q: ‘refKey:4e6b75c2-b527-4af0-a098-b2ba81706bdc’], headers: [‘Accept’: ‘text/plain’, ‘Content-Type’: ‘application/json’])

                assert response.status == 200
   
                consumer_service.buildInteractions()
            }


If everything is defined correctly, a JSON output will be generated in the file structure under the “target” directory.

The JSON output is contract of the API that you have defined in the consumer test, which is the API that the consumer consumes.

Introduction: API Contract Testing With PACT

PACT is a tool introduced to enable to testing of API contract between the consumer (the system that uses the API) and the provider (the system that produces the API). It is available in a couple of different languages. The version that I have been using is the JVM version in Groovy. This tool is very useful to detect API structure changes. It is good in the sense that we can test the structure  of the API irrespective of the data (we can just test the data type). It could be easily integrated into CI box as well.

 

Consumer Test

Consumer test defines the contract or structure of the API that the system uses. It can be used to enable TDD and in a situation when the system that consumes the API is built before the API exist. Upon successful consumer test run, a JSON file that mimic the actual API output will be generated under the “target” folder.

Provider Test

Provider test basically is a call to the actual API which then compares the actual API result with the structure which we expect (as defined in the Consumer test). It is built in the build script depending on which framework that we use, in my case it is called in the build.grade file.
I will include more specific examples with descriptions on the Consumer and Provider tests in my upcoming posts.

API Testing In General (PUT & DELETE cont.)

I have putting a few blogs about API testing in the past few months. I have been doing some research about how to test API, best tool to test API, etc and I guess there is no exact answer to this. However, SoapUI seems to appear in quite a number of searches. But I still do not choose SoapUI as it is not a pure Open Source tool.

So, I still think it all depends on the language of your API system, and there is a lot of built-in library to make a web request. All you need to do is to incorporate a unit testing framework to assert the result returned.

As I am working in a .NET environment, I have tried HttpClient and WebClient. I have used Curl for Ruby.

Example of POST and GET have been posted in the previous blogs.

Example of PUT is as below:

 

And DELETE:

 

Testing API in C# – WebClient – How To Get A Request

Below is how to Get a request using WebClient instead of HTTPClient

 

Testing API In C# – HttpClient – How To Get A Request

Back in days when I was working with Ruby on a MacBook Pro, I got Curl. However, now as I am using a PC in a .NET shop, I need to use compatible technology to test API – HttpClient is my solution.

Install the Web API Client Libraries via Package Manager Console

Install-Package Microsoft.AspNet.WebApi.Client

Sample code as below:


  • [Fact] is merely an XUnit notation indicating that this is a unit test
  • using creates an HttpClient instance which is only active within the scope
  • await suspends operation until the execution is completed
  • Method is declared as async as HttpClient methods perform I/O
  • ReadAsStringAsync() will read the output as string
  • JsonConvert.Deserializeobject<> will parse the string into the object model LanguageDictionary
  • You can use many methods from the HttpResponseMessage to verify the response from the API

 

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑