Getting the Performance Testing right is not easy. There are many factors which needs to be in place to ensure success from Performance Testing activity. Some pointers which if taken care can go a long way in success of Performance Testing:
The Goals of Performance Testing needs to be clearly identified. The approach to Performance testing and the Performance testing objective must be aligned to overall vision and goal of the project. It is important to understand the purpose of the application or system for which Performance Testing needs to be done. The Performance Tester needs to know the system intent , actual hardware and software architecture, Performance Test environment, comparison of test environment to actual production environment and characteristics of typical users of the application.
Identify the candidates scenarios for Performance Testing instead of trying to test all scenarios for Performance. The candidate scenarios can be identified based on time critical scenarios, business critical scenarios, most used scenarios, scenarios having financial impact, scenarios impacting claims made, peak traffic scenarios etc.
Establish Performance criteria, key points for establishing the Performance criteria are Business requirements, contractual obligations, key performance indicators, resource utilization, user expectations, SLAs, baselines from previous releases, competitor application performance etc.
While Designing the Performance Testing scripts it is important to design the real world simulation of the scenarios. Things to be taken into account would include determining the category of users and how they will use the application, what are their user behavior patterns, what are the activities they would spend time on, what are their expectations from the application, what are the peaks expected in the business, what are the scalability forecasts of the business, what are the technology constraints in which the application has to work, what are the endpoints in the system and how these end points impacted by increasing / decreasing traffic. These considerations will help in building the workload and workload profiling of the application.
Get thorough understanding of the Test environments, tools, utilities, load generators, controller, load balancers, test data, logs,monitors which need to be configured across various application layers, firewalls, network, what data to be monitored and collected from the Test runs.
Do couple of trial runs to ensure all the configurations, monitors, logs, connectivity etc. are working before actual Test execution. This will save lot of time in re-work, coordination and ensuring that correct data is collected during Test execution. Create a run plan for the Test execution, so that all the involved parties are aware of their tasks and contributions during the Performance Test run.
In Test execution phase, validate the configurations, tests, test data used and state of the environment. Ensure that all monitors are working properly and all the data is being collected as expected. Check the results of Test scripts, check the database to ensure transactions have been completed, check if the workload was properly applied, ramp-up and cooling down has happened as designed. Check the server logs to identify any alarms if present. Ensure all the dynamic elements are handled properly. Log the start time, end times and put timestamp on the scenarios, so that you have a clear picture which scenarios were executed at what time.
Once the Performance Tests are run, spend quality time in Analysis of the tests. Do not publish results without completing your Analysis. Analyze the test results by putting together data from all the layers, monitors, server logs, resource utilization data, time taken etc. Once the Analysis is done, summarize the tests with observations, concerns, questions and the inferences made from the test run. Know your stakeholders and what information they are interested in from Performance Testing. Represent the analysis visually so that people can understand easily.
Most importantly, Performance Testing cannot succeed in isolation. The Stakeholders, Business team, Development team, Infrastructure team, Database team and Test team all have to work in sync to ensure success of Performance Testing.