Defining the workload for software performance testing.
The importance of defining the appropriate workload to satisfy the performance project objectives has been emphasized in all of the basic performance project type descriptions. The reason why specification of the workload is key to these performance projects is that:-
It is useful to define workload within three basic categories:-
By way of example, lets consider the business function: Place Order, we could define the workload for Place Order as:-
We also need to consider one other key workload characteristic:-
Now consider the situation under which we have 7,000 employees working shifts and performing other functions. Out of these 7,000 any one of 700 could be logged on at a given time placing orders. Some of these 700 would be logged on all day and place orders at a steady rate whilst others could logon and logoff throughout the day and place one or two orders when logged on.
The question arising from the above scenarios is For the purposes (objectives) of the given load testing project do we need to simulate the potential user populations and their access patterns?
The answer to the usage access question is It depends on the requirements of the load testing project. For example: for an initial code stability test (looking for memory leaks or excessive CPU usage) it might not be important to recreate the user access patterns but if there was some elaborate security (access) algorithms that accompanied the Place Order function then the actual usage patterns might need reproducing to perform an accurate performance capability analysis.
In cases such as these, when specifying the workload requirements, the question of Is the difference material to the processing environment and\or the project's objectives should always be examined.
A basic workload specification for a stability test.
The type of workload specification questions (noted above) need to be asked as issues such as caching versus disk reading will come up when repeatedly reusing the same data. In all cases the material difference of the workload specification options needs to be weighed against the load testing objectives.. The answer to this question is a pragmatic one that depends on what you are trying to conclude from your performance testing project.
The workload specification versus the workload implementation in a given load testing tool.