How to make test automation work for you
June 23, 2023
Frederique De Winter
Today’s growing demand to deliver qualitative software faster requires organizations to search for solutions in Agile, Continuous Integration and DevOps methodologies. Test automation is an essential part if this process. But in a lot of cases testing (manual and automated) often becomes a bottleneck. The reasons can be various, but very often the problem begins with the choice of tooling to support this process. So how can we tackle this? How can we avoid test automation becoming a bottleneck for the organization?
First things first – a structured test process
For starters, you need a structured test process. The reason for this is obvious, test automation should fit into this testing process. So, first think about what you want to test and how you want to test it.
Once this test process is in place, it’s time to start thinking about optimizing it by introducing test automation. Luckily there are plenty of test automation tools, frameworks and libraries available. Each and every one of them has its own advantages and disadvantages, but not every solution will suit your project’s prerequisites. And this is where the problem begins.
In order to choose the most appropriate tool, you will need to closely examine the project’s specifics. The most important questions you need to ask yourself are:
- What do you want to achieve with test automation?
- On which levels do you want to test?
- Do you want to perform unit, component, integration or system testing?
- Which kind of tests do you want to execute on each level?
Once decided, you can start thinking about automating these tests.
Determine prerequisites
Maintaining the quality of an application by delivering a bug-free product is crucial for the success of any project. Automated testing can help improve the quality of a project and increase the testing scope and depth. But to achieve this, we must choose the right tool for the job. Therefore, we need to get a deep understanding of the project requirements such as project type (web/desktop/mobile), scope of the project, existing team’s strength on code language, … before you start the process of choosing an automation solution. No tool is good or bad, the Return Of Investment (ROI) of any tool or solution relies on the need, the prerequisites. To determine these prerequisites, you can ask yourself some questions.
What to automate
- Does it concern a web, mobile or desktop application? Or is it a combination of multiple technologies?
- Do you need functional and/or non-functional testing?
- Which type of devices should be supported? Or maybe which type of browsers?
- What amount of test cases need to be automated?
- How often do you want to run your automated test sets?
Who will automate?
- What is the technical knowledge of the test automation team?
- Which programming language is preferred?
How to automate?
- Do you want to work BDD- or TDD-style?
- Are you working in a DevOps environment?
- Does the automation solution need to integrate with CI/CD?
- On which type of OS does the automation solution needs to run?
- Which type of report do you need? Do you need an .html report or rather .pdf?
- Do you need to integrate with a test management tool? Or even with other tools like cloud testing platforms, device farms, Slack, …?
This is just part of the questions you need to ask yourself before selecting a tool.
Automation solutions
Based on the prerequisites we can already make a list of possible automation tools. But there is still one important question to be asked. What is the available budget? Depending on the answer you can determine whether to go for commercial tools or free open-source solutions.
Commercial tools
The big benefit of commercial tools is that you will be able to start automating almost immediately after purchasing a license. These tools have a user interface which should make it much easier to automate a test, even if your technical knowledge is limited. But still take into account that there is a high possibility that not everything can be handled with the UI and very often some scripting and technical knowledge will be required.
Free open-source solutions
The fact that these open-source solutions are free, also comes with a cost. Not a license cost, but the cost of setting up these solutions and scripting the automated tests. For this kind of solutions a deep technical knowledge is a requirement to be able to get the most out of it.
In a lot of cases, we see that commercial tools are often too expensive and open-source solutions require too much technical knowledge. Fortunately, there is a solution in between: a test automation framework.
Automation framework
A test automation framework is a defined, extensible support structure, which consists of a combination of best practices. These designed to help in developing, maintaining and executing automated tests more efficiently. It requires development time to implement such a framework, this is a one-time cost instead of a recurring license cost. The major advantage of an automation framework is that it can be fully customized, based on the needs (prerequisites) of the organization. They offer a lot of flexibility in all kinds of aspects:
- Build your own reporting and report in whatever output that you prefer
- Create integrations with other tools (i.e. test management tools)
- Determine how to handle test data
- Choose your own programming language and IDE
- Determine if and how you want to store test results
- Build a less technical layer for creating and maintaining tests
- And many more …
Once developed, a framework must facilitate a standard way for modifying, adding, and deleting test scripts. It should also provide scalability and reliability with less effort. This kind of framework also gives you the benefit that you, as an organization, have everything under control. You are not depending on any tool vendor. Developing your own automation framework, customized to your needs, will come with an initial development cost, but it will make you more futureproof.
Conclusion
Setting up a test (automation) strategy isn’t something you do overnight. It takes time, effort and for sure some insight in the different matters. We can help you every step of the way. From assessing your current IT/Test organization with our BrightScan to implementing suitable automation tooling.
As we are vendor independent, we can give you objective advise on which tools would(n’t) meet the organizations demands. If nothing really suits your needs, there is always the possibility to build a personalized automation framework. At Brightest we believe real quality requires a holistic view. Cost-effective software quality requires more than just testing skills (read more on this topic in our Whitepaper on “How to set up cost-efficient software testing?”). Together with you, we look at the complete picture and focus our efforts on high return on investment.