Automation Tool Selection Criteria
Selecting right automation tool for the application is always a challenge. there are various parameters we need to consider during selection.
in this post we will focus on factors we need to consider while selecting the right automation tool for application under test.
On broadly we can classify the application into Three categories
- Desktop Applications
- Website/Web applications
- Mobile Applications
each category requires specific automation tools. also it depends on the application which type of automation tool it can supports.
lets see what are common factors needs to consider while selecting the automation tool.
- Development Platforms of the application
- Support for Operating Systems
- Support for Internet Browsers ( for web and mobile applications )
- Support for various of testing in easy manner
- Required less programming skill
- Strong Record and playback mechanism.
- Support for Cross browser tests ( for web and mobile applications )
- Strong in Object and Control recognition
- Support for Various Data Resources
- Running Automated Software Tests and Synchronization
- Provide inbuilt Verification features
- Reports ( Report generation and Export of Reports)
- Integration with other testing tools
- Command Line and OLE Automation Support
- Automation Tool Support
So i think this list well enough for selecting right automation tool. lets see the details of each points mentioned above.
i am assuming that you know how automation works?. if you don’t know please comment i will post separate article on it.
1) Development Platforms of the application
Many Automation tools supports specific development platforms say .net, Java, etc. and limited versions of it. so its become very hard to select the automation. so while selecting the automation, make sure the development platforms may not change or select the tool which supports lot of platform and their versions so that same automation tool can be used for other projects as well.
2) Support for Operating Systems
Try to select the automation tool which supports all famous Operating systems. many times its found that we need to run automation testing suite on different platform if we integrate it with some build systems etc. so make sure that you have good vision while selecting the automation tool.
3) Support for Internet Browsers ( for web and mobile applications )
while selecting the tool make sure that it supports all major internet browsers. while automating the web application or web based mobile application its very common scenario where we execute the tests on the different browsers as manually it is not possible to do so. Selenium is very good and having great range of supporting browsers than any other tool.
4) Support for various of testing in easy manner
Moving towards the functionality of the Automation tool then it should easy to use and support various of type testing. like functional, UI, data driven, etc. and off course in easy manner so that tester can create test cases really fast and more productive. many automation tools support lots of testing types but its hard to use or take lot of time to automate the scenarios. so always focus on Ease of use while selecting any automation tool.
5) Required less programming skill
It is one of most important factor which we need to consider while selecting the automation tool, Automation tester should not required high programming skills to automate the scenarios. before selecting the tool make sure automation tool provides good amount of in build libraries by which tester can easy automate the test cases although tester has less knowledge on the programming side.
6) Strong Record and playback mechanism
As above this is also an important factor to consider. most of the automation tools provide record and playback mechanism by which automation tester can record the script and after editing it, same script can be used as test case. it is found that many recording automation tool are not able to record certain actions by which automation tester needs to add these tests later and it come time consuming job. so make sure that the application which you are going to test, the automation tool can identify all features and it can be recorded.
on the practical fact, non of the automation tool recognize all features, you need to spend some to rework on the recorded scripts.
7) Support for Cross browser tests ( for web and mobile applications )
for many complex web application testing we need to make sure the automation tool can support the cross browser testing. in cross browser testing, oftentimes during test execution website site url gets changed. take a simple example of banking applications, or any e-commerce site. so always make sure that tool should support cross browser tests.
8) Strong in Object and Control recognition
This is most import factor while selecting the automation tool, object and control recognition. select the automation tool which can identify all your objects and controls. i will say 90% of objects should be recognize by tool itself. this make automation tester life easy.
9) Support for Various Data Resources
in most of the application, there are some test for which we need to run for large number of data set. it is most commonly found that automation tester uses, csv, execel, txt, database and xml files to store such data and they used in their tests. so it is very import that automation tool support the various data resources so that it is easy to do the data driven testing.
10) Running Automated Software Tests and Synchronization
many times it found that during test execution tests failed due to un-synchronization. like, timeout, late response from system etc. also while test execution on distributed environments. while automation tool evaluation make sure that tool gives good respond to such things. tool should handle it properly which will leads to less script failure in the automation.
11) Provide inbuilt Verification features
When we talk about the feature of the automation tool, i will say, tool should provide as many as verification feature to save the tester time to automate the test case. many automation tools are doing the same and getting more and more popularity. there are features i like mention are text verification on label, text boxes, text area etc., image comparison, file comparison, database checkpoints etc.
Make sure that automaton tool has decent logging feature, this can be used while automating the test cases and analyzing the test case failure.
13) Reports ( Report generation and Export of Reports)
Every automation tools are having their own reporting styles and most the report contains all data about the test case passed, failed, skipped etc. along with the logs. most of the time automation tester created own reports as per the need of the project so this factor really does not matter to me at least but yes, i really want export functionality on the reports so that i can tweak my reports as per my requirement.
14) Integration with other testing tools
while selecting the tool make sure that automation tool has integration with other tools, like build tools, test case management tools, defect management tool by which we can integration and automate the complete QA workflow.
15) Command Line and OLE Automation Support
To take the advantage of automation tool, most of the company put automation to verify the build and to make automatic, build engineer write script to invoke automation suite after successful build made on server. whenever we want to trigger automation suite without human intervention, we write the script and if automation tool has feature to invoke the test automation from command line benefits the automation user.
16) Automation Tool Support
Make sure that automation tool has good support and learning material available on internet to help you out.