Performance testing easily explained

May 26, 2023

Koen De Wilde

Your soccer team’s championship celebration, the toll roads in France during the summer and online platforms in times of the Covid crisis. They all have a similar problem. And how can running a performance test avoid this? Let’s find out!

Situations we all know

For these three examples you can estimate normal (load), high (volume) and extremely high (stress) consumption/use. Let us also assume that functionally everything is working well.

 

Where does it go wrong?

If you ask us, at the estimation for more than normal usage/consumption, i.e. volume and stress. Since you’re not here for soccertalk or travel tips, we suggest zooming in on the school platforms. A problem some of you may have faced in the recent years.

In the case of school platforms, the maximal usage during Covid times may have been barely predictable in 2020. Therefore, we suspect that in the past tests have been done only with a lower load. Now, due to the Covid crisis, new parameters are available to re-run the performance.

 

What do we mean with a performance test?

A performance test consists largely of estimating what is normal, high and maximum usage of an application. But also, what response time can be expected. And response time is the time required to perform an action/transaction in the application. Important is that this remains acceptable to a user. If a user must wait too long, he/she gets the feeling that the application is no longer working correctly. These performance tests are part of the non-functional requirements of an application or system. In other words, not what an application does (functional requirements), but how it does it.

Everyone will remember when, a decade ago, websites for concert tickets consistently crashed when popular bands announced their stadium tour. Since then, these websites learned to adapt their technical infrastructure to the (measured) load on their applications. By measuring and adjusting, nearly all concert websites are largely performant. They let you wait, for example, in a neatly organized queue, until it’s your turn.

These websites are lucky to have quite many iterations of maximal load on their system. Every time a world star announces a concert, there is a new maximal load/stress measurement on their system (= monitoring in a live environment). This allows them to adjust each time. In case you don’t have this luxury, like the school platforms, it’s better to preventively test and make estimations (= performance test in a test environment). Here, the test environment best approximates the final live environment as much as possible.

 

Different types of tests

By fine-tuning, customizing and parameterizing automatic test scripts, different load simulations can be run on the Application Under Test (AUT). During these tests, different test-types can be simulated:

  • Load test
    Measuring performance of the AUT while increasing the load on the application. This means more concurrent users and/or (trans)actions. The AUT is monitored on response times, memory use, etc. All action/load is within the normal circumstances and expectations.
  • Volume test
    Volume testing defines how the AUT reacts to higher load. There are more users and more actions. Here the CPU usage, Disk usage and memory leaks are monitored as well.
  • Stress test
    With this testing the AUT is over-encumbered, overabundance of users and multiple actions. The AUT has more load than it was designed for. The goal is to measure the stability of the AUT. When does the application fail? How and when does the AUT recover from the stress testing?

These are the baselines of performance testing and their definitions. Depending on the AUT and infrastructure (Cloud), variations in performance testing are possible. Additionally, performance tests can be executed very early in the design process of the AUT. Whereby the type and goal of the performance test changes according to the stage of development. 

Another necessary phase in the execution of performance tests, is the analysis of the result (data and results). Some tests might be executed again with different parameters to get to better conclusions. From these results follow improvement suggestions for different, non-functional aspects of the AUT.

 

Get assistance from a professional

Brightest can help companies like this with the deployment and execution of their performance test. In the past we helped, among others, a retailer set up tests for their web-platform and online store. Retailers have several peak moments like Christmas, Black Friday, etc. We also invest strongly in R&D and the expansion of our Performance Testing Service. For example, seven of our consultants became Octoperf Certified. 

Let’s work together

Interested in how performance testing can help your organization?

Contact us