Agile automated testing
July 14, 2023
Frederique De Winter
Nowadays many companies are working Agile or are switching towards it. Agile allows you to deliver new features faster and more efficiently. But how can we assume the quality of those new features?
What is agile (testing)?
The Agile methodology is all about delivering incremental improvements to your product and can be achieved through processes like Scrum. It splits the development of the product into small sprints. At the beginning of each sprint, you need to prioritize the features you will be working on. At the end, you need to check if you have delivered what the user needs. The results are many small incremental product releases, which must be tested for quality and stability.
In the past, when using a waterfall process, testing happened only once you had completed the entire development of the product. Nowadays, we see many specification changes during the development process. This also means that the QA team is faced with testing against a constantly shifting target. So, Agile requires testing to be done continuously throughout development.
In other words, you’ll need your testing to be Agile as well. This implies that you should be able to test the whole product during every development sprint. In most cases, this will only be feasible by using test automation. The computer will interact with our UI and will replicate the actions of a real user. Typically, these types of tests will rely on tools like Selenium, Cypress, …
Agile automation challenges
The Agile methodology is the fastest approach to turn an idea into reality. Especially for dynamic projects that are constantly changing. The biggest advantage of working Agile is the capability of continuous iteration. But this is also the root cause of many major obstacles.
Test creation
Developing an automated test can take several hours or sometimes even days, especially when dealing with complex UI applications. That is because each test needs to be implemented, tested and debugged. Adding cross-browser and cross-platform support can make this process even slower. When testing new features during a sprint, the time needed for test creation might become a bottleneck.
Regression testing
Agile methodology’s nature is meant to adapt to continuous changes. For every single update, the application’s code-based and UI layers must be tested again to assure they are functioning correctly upon these new changes. This results in a massive amount of regression testing for QA teams to perform.
Test debt
During each sprint you only have a limited amount of time. That time needs to be split up between creating tests for the new features that are developed within this sprint, maintaining existing regression tests and executing and analyzing all the results. Thus, test maintenance for the regression tests will increase with each sprint and eventually the team won’t be able to complete all the work they need to do. This ever-increasing backlog of overdue test maintenance is what we call test debt. Agile testing requires your team to follow an Agile approach, breaking down their work based on sprints of the development team. During every sprint the test team needs to prioritize between automating new features, maintaining existing automated scripts and running tests manually. But test debt means that something has to give and the test team ends up making though decisions simply to meet the sprint goals.
Maintaining
Automated UI tests are very fragile. Since for many applications the elements are missing a unique id, we often need to rely on CSS or xPath selectors. Our automated test uses these selectors to identify the correct element to interact with. The downside of these types of selectors is that they can change unpredictably whenever the UI is updated during sprint. If such a selector changes, there are two possible outcomes: the test breaks because the selector is broken and cannot find an element or the selector selects the wrong element to interact with and the test fails several steps later. Each time the UI of your application changes, you also need to maintain your CSS and xPath selectors.
At Brightest we have developed our own Brightest automation framework. Which is designed as a future-proof solution and is independent of any vendor.
Are there any solutions?
System and service level
In an Agile workflow the UI is bound to experience many changes. In terms of UI maintenance, test automation tends to be very time-consuming. To keep maintenance costs down and enhance the overall coverage, automation needs to be executed as much as possible on system and service level. Creating, executing and maintaining an automated API test takes much less time than an automated UI tests. That’s why we must try to keep the number of automated UI tests as low as possible.
Unique ids
When you create an automated UI test, make sure to use as much unique ids as possible. It will result in faster test creation, less failures and less maintenance when small UI changes are done to the application.
Automation framework
Use an automation framework. An automation framework is a combination of guidelines and structured code. When the framework is well-designed, it will have many benefits:
- A well-designed framework will encourage re-usability across your tests, which will result is less maintenance.
- A keyword-driven framework will speed up the process of creating new tests for new features during the sprint. The keyword-driven framework in combination with the framework guidelines, will result in a readable and understandable way of writing a new test. And that even for less technical people.
- Separating the framework from tests and test objects will result in less maintenance. Since any change in a test will have much less impact on the test objects and the framework itself. The same goes for any change in any of the test objects, this will have no impact on the tests and the framework itself.
- A standardized way of working across all teams.
A framework has many benefits in an agile environment, but it still needs to be developed. It will take time, but it will only be a one-time effort. Once implemented in the correct way, the same framework can be used for all projects without any extra effort.
Brightest Automation Framework
At Brightest we have developed our own Brightest Automation Framework, which is designed as a future-proof solution and is independent of any vendor. This framework is extendable and can be customized depending on your needs and requirements. No fairy tales, complex applications require complex automation. With this framework, we try to minimize the complexity for the tester who’s creating the tests. It’s made based upon all our previous experiences and will for sure kickstart your test automation!