Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Year: 2014 (page 1 of 2)

Evolution Of Software Testing

I was lucky enough to be able to present in the Melbourne Selenium Meetup which was held at Carsales.com.au on the 11th of December, 2014. 
Below is the script of my speech with the title Evolution Of Software Testing.

Introduction

Thanks to Ray for inviting me to the MeetUp. I am definitely not the most experienced here, just to share with everybody the changes in testing that I have observed in the past 10 years, which you may be experiencing too and my personal experiences in the journey.

Agenda

I would like to share my experience in terms of these 4 areas.
(1) Transition from traditional to Agile
(2) Importance of automation 
(3) Battling between time and quality in a world when we need to deploy fast 
(4) What do we need to do to keep up in a rapid changing industry

Waterfall to Agile

Most of our changes started after the adoption of Agile SDLC over Waterfall

Testing is part of software engineering

Testing no longer is a phase in SDLC. It is part of software engineering. What does that mean?

Bug catching at every phase

Bug catching is not done at the final phase before deployment. Testing starts at early phase, during planning or design, before development starts.

Real Life Scenario

My experience with the transition from Traditional to Agile is that it helps to deliver better quality results and on time as it eliminates the back and forth bug catching and fixing which will cost more towards the end of a development phase. Testing bit by bit is gives a much better result than testing in a big chunk. This also helps to ensure that the product is what the stakeholder wants.
I heard about a real life scenario where Company X invested in a 4 year traditional software project. When the stakeholder 1st time sees it, it was a shock as it was not the product they expected.
During Waterfall, testing is the final phase before production and testers are always being pressured to test fast in order to deliver on time. My worst experience is to receive a project that would need a few days of testing at 3pm and needs to be deployed the next day. As a result of that, we have to work overtime, or take out features that have show stopper bugs. PMs tend to give a lot of pressure to the testers back in those days.

Get rid of the documentations

We no longer need to write piles of test cases or test plans.

Automation is the key

Automation is our documentation. Yes, we need to learn to code. Test cases or test plans are converted into automated tests.
No technology is perfect without a testing framework. Every new technology or language comes with a way to test it.
Examples.
Web applications: Selenium
Mobile: Calabash, Appium, Robotium
.NET: Nunit, Xunit
Java: Junit
Angular JS: Protractor

Choosing the right tool

How to choose the right automation tool? These are the key criteria to consider. 
Most popular used
Large community support
Open source vs Commercial
Skills of the team
Personally I prefer Open Source tools as they are more fun to work on. 🙂 Or even custom built. For example, we use .NET library Web Client to test API and Web services

Real Life Scenario

So how do we start from doing manual testing to automation? Take baby steps. Start writing BDD scenarios instead of test cases. Then learn how to use Selenium to drive web browser. Then try more complex code manipulation before moving on to the framework level in making sure the test framework is reusable, scalable, maintainable. 
As you expand your Selenium usage, you’ll realize that we will face some challenges in certain areas such as pages with lots of AJAX calls. In that case we will need to have workaround these scenarios, like checking that all jQuery has been loaded.
Another Selenium problem is Element No Longer Attached To DOM. In this case, we will need to have a while loop and try to find the element with a catch for StaleElementException.

Test = User + Design

We need to use the product as a user and also understand the design to be able to think of the possible loop holes 
In the older days there is less emphasize on tester to be know the software design architecture. But in Agile, as we are involved since the earliest phase of SDLC, we will are expected to know to pick up not merely software bugs, but also requirements and design issues.
Knowing the software design helps to pick up edge cases. For example:
If we know that the database field has a size of 50 varchar, we can try to input >50 chars into it and see what happens. Will it crash or truncated?
If we know that a CSS file is used in multiple location, we will need to test multiple locations if it is changed

Time vs Quality

Nowadays we need to deliver software fast. Deliver, get feedback, or fail fast is the best practice. So how can we do that without sacrificing quality? I used to fight with our PM back in he old days in order to get the best quality product out the door. But nowadays that does not work anymore because time = money for business and as a QA, we need to work towards the same goal too (management will like this).
We can deliver software with known bugs. We do this by understanding the critical path of the system usage and ensure that most red route are working correctly, while some bugs can be left in some feature that has minimal usage. We can use data or user analytics to understand which are the most visited pages, most used features, etc to help us make the decision.
My personal experience is when I started at my current job where we do not automation at all and testing happens after deployment. Our product is complex and I am having time pressure to put some automation in place.
How we go about to solve this problem is to find the top 20 pages which equals to 90% of usage. We then analyze the data and pages and put them into a logical user flow. With that we then translate that into BDD language and automate the scenario based on it.

How To Stay Alive

So software testing has changed so much and it is constantly changing just like everything is. How can we keep up with this?
We need to constantly learn. Don’t worry if there isn’t enough training budget (management will like me once more), as learning doesn’t always cost $.
There are many different channels that we can learn which are free. These are some of my personal likes. 
My personal experience with learning are: Examples:
New tool: Appium by Jonathan Lipps at GTAC 2013
New ideas: Take flaky tests from CI. Put is back when they are stable. Critical tests should never be taken out by Ankit Mehta at GTAC 2014
New ideas: Turn off a feature that is buggy by Ankit Mehta at GTAC 2014
I then I like to Blog about what I have learnt. Its not just serves to share with people what I’ve learned, but also to keep a note on what I’ve learnt.
We also need to get our hands dirty and get some hands on experience with what we learned. We can only really learn something when we work on it and gain experience from it. When I face problems I will post questions onto forums like StackOverflow, or Google forums. 
Lastly, share and give back to the community what we learnt. Try to help people out there that are learning just like us.
And most important of all is to love what do! As what Steve Jobs says.Thank you.

Appium: Introduction & Getting Appium Working for iOS App

Introduction

I came across Appium as an open source tool for mobile testing in the Google Testing Conference 2013. The details from the presentation can be found GTAC 2013 – Appium: Automation for mobile apps.
The previous open source mobile automation tool that I had experience with are quite restricted in terms of its whether for merely iOS like Frank or Android like Robotium.
Appium is a specialised testing tool for both native and hybrid iOS and Android apps, including web applications on mobile web browser.
In my coming posts, I am going to share with you my experience in spiking out the tool in both Mac and Windows environment.

Appium working in iOS

Install

  • Get a Mac. My colleague of mine has tried to set up a virtual machine to run XCode on Windows but don’t seem to have much luck.
  • Install node.js as it is needed to “fire up” Appium, otherwise you can download the Appium app
  • Download Appium as illustrated from the screen shot taken from the Appium website
You are able to find sample test code written in many different languages. The good thing about Appium is that it is not language dependent, so feel free to choose any language that you’re more familiar with. In my case, I’ve chose Ruby.

Run test for iOS App

  • Start Appium by typing appium & in the Appium folder
  • Start iPhone simulator by going to Xcode->Open Developer Tool->iOS Simulator. Under iOS Simulator->Hardware->Device, you can select the device type. Eg: iPad or iPhone

  • Ensure we set the path of app correct in sample-code/sample-code/examples/ruby/cucumber_ios/features/support
  • Go to the sample-code folder and run the Cucumber test

How To Automate Web Services Testing

Testing web services is similar to testing API. Basically, all we need is a framework that:

  1. Calls the web services
  2. Checks the response
  3. Asset the value returned accordingly

The framework we choose is dependent on the framework of the web service itself, for example, if the web service is written in Java, we would want to use Java to test, if its written in .NET, we would use C# to test, etc.

In the example above, we:

  • Create a class extending WebClient

public class ExtendWebClient : WebClient

  •  Create a variable of class ExtendWebClient with overridden timeout

var client=new ExtendedClient(_TIMEOUT)

  • Set the header of the request

client.Headers.Add(HttpRequestHeader.ContentType,”text/xml”)

  • Set the XML data to be sent as string request

string request =”your XML here”

  • Calls the web service by passing the web services URL and XML (string)

 client.UploadString(new Uri(String.Format(“{0}/ADAuthenticate/ADAuthenticate.asmx”, _WEBSERVICEBASEURL)), request);

  • Web client will automatically return a success code (500) and error code otherwise

API Testing In General (PUT & DELETE cont.)

I have putting a few blogs about API testing in the past few months. I have been doing some research about how to test API, best tool to test API, etc and I guess there is no exact answer to this. However, SoapUI seems to appear in quite a number of searches. But I still do not choose SoapUI as it is not a pure Open Source tool.

So, I still think it all depends on the language of your API system, and there is a lot of built-in library to make a web request. All you need to do is to incorporate a unit testing framework to assert the result returned.

As I am working in a .NET environment, I have tried HttpClient and WebClient. I have used Curl for Ruby.

Example of POST and GET have been posted in the previous blogs.

Example of PUT is as below:

 

And DELETE:

 

Testing API in C# – WebClient – How To Post A Request

Testing API in C# – WebClient – How To Get A Request

Below is how to Get a request using WebClient instead of HTTPClient

 

Testing API In C# – HttpClient – How To Get A Request

Back in days when I was working with Ruby on a MacBook Pro, I got Curl. However, now as I am using a PC in a .NET shop, I need to use compatible technology to test API – HttpClient is my solution.

Install the Web API Client Libraries via Package Manager Console

Install-Package Microsoft.AspNet.WebApi.Client

Sample code as below:


  • [Fact] is merely an XUnit notation indicating that this is a unit test
  • using creates an HttpClient instance which is only active within the scope
  • await suspends operation until the execution is completed
  • Method is declared as async as HttpClient methods perform I/O
  • ReadAsStringAsync() will read the output as string
  • JsonConvert.Deserializeobject<> will parse the string into the object model LanguageDictionary
  • You can use many methods from the HttpResponseMessage to verify the response from the API

 

Chrome Driver Bug – Disable Developer Mode Extensions Pop Up

There is a bug in the Chrome Driver where you will get a Disable Developer Mode Extensions pop up occasionally in your test.
I still cannot work out why we started to get this problem as we have not upgraded the version of the Chrome driver.
The even more odd thing is that I found out that the pop up appears on bigger pages with lots of data and by cleaning up the data is able to temporarily get rid of the pop up.
However, the proper solution is to have the following line in your code:

options.AddArguments(“chrome.switches”, “–disable-extensions”)

So, the final code looks like this:
    var options = new ChromeOptions();
    options.AddArguments(“chrome.switches”, “–disable-extensions”);
    var driver = new ChromeDriver(_CHROME_DRIVER_PATH, options);
However, you might get this message on the browser while you run your tests:
But don’t panic, because everything will work perfectly without the pop up message!

Selenium Webdriver: How To Deal With Pages With Complex Ajax Calls

If the web page that you testing consists of complex ajax calls, you may be facing the problem where the tests will fail occasionally due to incomplete ajax calls returned.

To deal with this problem, I have added a function in my test:

driver.WaitForAjaxLoadComplete();

And what the function does is:

 public static void WaitForAjaxLoadComplete(this IWebDriver driver   {

            IWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(60.00));

            wait.Until(driver1 => (bool)(driver1 as IJavaScriptExecutor).ExecuteScript(“return jQuery.active == 0”));

        }

Basically, it tells Selenium to wait until all jQuery calls are completed before the next actions.

Selenium Webdriver: How To Deal With “WebDriverException: Element is not clickable at point (x, y). Other element would receive the click:”

Recent task of me is to make the Selenium tests reliable. Functional tests have the following drawback:

  • Update of tests are required when DOM changes
  • Selenium work differently when you use different web drivers
           Eg: A checkbox is visible when you use Firefox, but hidden in Google driver
  • Depending on how the web page is constructed, you may get inconsistent behavior with the tests.
          Eg: Element is clickable now but not when you run it again
In my previous post, I talked about the “Stale element exception” which we have to find some other creative ways to get around. Please note that there are multiple ways to get around that problem which I will share with everybody in my future post.
Today, I am going to talk about a specific problem I have when using the Google Driver –
WebDriverException: Element is not clickable at point (x, y). Other element would receive the click
 
 
The solution is:
Instead of using the usual Click method:

driver.FindElement(By.Id(“id”)).Click;

We use the Actions method:

IWenElement element=driver.FindElement(By.Id(“id”));

Actions actions = new Actions (driver);

actions.MoveToElement(element).Click().Perform();

« Older posts

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑