Performance testing easily explained
May 26, 2023
Koen De Wilde
Your soccer team’s championship celebration, the toll roads in France during the summer and online platforms in times of the Covid crisis. They all have a similar problem. And how can running a performance test avoid this? Let’s find out!
Situations we all know
Your favorite soccer team's championship celebration
Situation
Your favorite soccer team becomes champion on the last day of the season, on their own field.
Problem
Two hours after the match, in full celebration, the beer ran out. An estimate had been made of how much beer would be drunk, based on previous victories at the club.
Impact
Unhappy partygoers: “Surely they could have seen that coming?!” The club rushed to call surrounding bars to bring in extra kegs of beer.
Need
Better and more realistic estimates of what the beer consumption might be at a championship celebration. Possibly arrange with the brewer to standby at the last game of the season in case additional beer be needed.
The French toll roads during summer
Situation
‘Black Saturday’ on the road due to a combination of departing and arriving vacation traffic. In France the lanes are clogging up and at toll points it is a miserably long traffic jam.
Problem
The road network cannot adequately handle all the extra vacation traffic. At the toll booths bills and coins are exchanged and credit cards are entered as quickly as possible in exchange for free passage for the next 200km.
Impact
Unhappy travelers: “Surely they could have seen that coming?!”. Vacation mood in the car drops below zero. Cut-through traffic along the national roads which just shifts the problem.
Need
A better spread of summer vacation periods across Europe (already partly in place), extra lanes, if possible, a rush hour lane and smoother traffic flow at toll booths.
Online school platform during Covid
Situation
Learning platforms that are used as additional tools to everyday teaching suddenly become the heart of education. They must provide documents, videos, PowerPoints and exercises to children at home.
Problem
Too many simultaneous movements, users and actions on these platforms. This causes longer response time and timeouts, also known as ‘the website doesn’t work’.
Impact
Frustrated parents, teachers and students: “Surely they could have seen that coming?!”
Need
Better and more realistic assessment of what the online usage might be in the event of a (Covid) crisis. Potentially an arrangement with the Cloud Provider to expand capacity when necessary.
For these three examples you can estimate normal (load), high (volume) and extremely high (stress) consumption/use. Let us also assume that functionally everything is working well.
Where does it go wrong?
If you ask us, at the estimation for more than normal usage/consumption, i.e. volume and stress. Since you’re not here for soccertalk or travel tips, we suggest zooming in on the school platforms. A problem some of you may have faced in the recent years.
In the case of school platforms, the maximal usage during Covid times may have been barely predictable in 2020. Therefore, we suspect that in the past tests have been done only with a lower load. Now, due to the Covid crisis, new parameters are available to re-run the performance.
What do we mean with a performance test?
A performance test consists largely of estimating what is normal, high and maximum usage of an application. But also, what response time can be expected. And response time is the time required to perform an action/transaction in the application. Important is that this remains acceptable to a user. If a user must wait too long, he/she gets the feeling that the application is no longer working correctly. These performance tests are part of the non-functional requirements of an application or system. In other words, not what an application does (functional requirements), but how it does it.
Everyone will remember when, a decade ago, websites for concert tickets consistently crashed when popular bands announced their stadium tour. Since then, these websites learned to adapt their technical infrastructure to the (measured) load on their applications. By measuring and adjusting, nearly all concert websites are largely performant. They let you wait, for example, in a neatly organized queue, until it’s your turn.
These websites are lucky to have quite many iterations of maximal load on their system. Every time a world star announces a concert, there is a new maximal load/stress measurement on their system (= monitoring in a live environment). This allows them to adjust each time. In case you don’t have this luxury, like the school platforms, it’s better to preventively test and make estimations (= performance test in a test environment). Here, the test environment best approximates the final live environment as much as possible.
Different types of tests
By fine-tuning, customizing and parameterizing automatic test scripts, different load simulations can be run on the Application Under Test (AUT). During these tests, different test-types can be simulated:
- Load test
Measuring performance of the AUT while increasing the load on the application. This means more concurrent users and/or (trans)actions. The AUT is monitored on response times, memory use, etc. All action/load is within the normal circumstances and expectations. - Volume test
Volume testing defines how the AUT reacts to higher load. There are more users and more actions. Here the CPU usage, Disk usage and memory leaks are monitored as well. - Stress test
With this testing the AUT is over-encumbered, overabundance of users and multiple actions. The AUT has more load than it was designed for. The goal is to measure the stability of the AUT. When does the application fail? How and when does the AUT recover from the stress testing?
These are the baselines of performance testing and their definitions. Depending on the AUT and infrastructure (Cloud), variations in performance testing are possible. Additionally, performance tests can be executed very early in the design process of the AUT. Whereby the type and goal of the performance test changes according to the stage of development.
Another necessary phase in the execution of performance tests, is the analysis of the result (data and results). Some tests might be executed again with different parameters to get to better conclusions. From these results follow improvement suggestions for different, non-functional aspects of the AUT.
Get assistance from a professional
Brightest can help companies like this with the deployment and execution of their performance test. In the past we helped, among others, a retailer set up tests for their web-platform and online store. Retailers have several peak moments like Christmas, Black Friday, etc. We also invest strongly in R&D and the expansion of our Performance Testing Service. For example, seven of our consultants became Octoperf Certified.