Pre-launch performance testing is important and should be high on the priority list of every site and app. Failure to carry out load testing can leave your website or app vulnerable, slow or at risk of crashing. You can spend months designing, building and marketing your site and app but if you don’t carry out load testing and stress testing then you risk losing customers, capital, and your online reputation.
In this article we’ll take you through why performance testing is needed, including examples of load testing and performance testing types.
What is Performance Testing?
Performance testing is carried out to establish how well a system is coping in terms of responsiveness and stability when faced with a particular workload. Performance testing allows you to see whether your website or app will slow down or crash completely once it receives a substantial amount of traffic that requires it to carry out tasks.
Performance testing will help your team identify any weak spots before your product is live and it’s fail safe way of ensuring that your site will still up when the going gets tough.
Why do businesses need performance testing?
The e-commerce sector is a highly competitive industry, if you’re business doesn’t deliver what a user needs then you can be sure that another site will. 47% of consumers expect a web page to load in 2 seconds or less and 40% of people abandon a website that takes more than 3 seconds to load.
Long wait times, delays, errors and service interruptions will frustrate users and you’ll lose their trust. It hurts the online image of your brand and when your website is down your business is literally losing money every second. A 1 second delay in page response can result in a 7% reduction in conversions. This means that if an e-commerce site is making $100,000 a day, a 1 second page delay could potentially cost you $2.5 million in lost sales every year.
It might seem obvious that a fast and responsive website that can keeps visitors happy and engaged would be high on the list of every business’ goals. Surprisingly, a lot of high end brands fail to carry out performance testing before launch and end up paying the consequences (we look at some well known examples further below).
Who needs performance testing and when should it be carried out?
Best practice dictates that all sites and mobile apps should carry out performance testing before launch to ensure that they will be able to handle the expected load and that response time is quick enough.
Perhaps the most precarious time for a website is when a business is experiencing growth and garnering new users. This is a crucial time for the business and a website crash could destroy its visibility and online reputation, apart from the obvious short time hit to sales.
In an ideal world your website will perform well throughout the year but there are certain milestones that you should be prepared for and carry out performance testing in the run up to them. For example when you are offering discounts or launching a big promotional sales like Black Friday then you should expect more strain on your website. Other examples include advertising campaign launches or when you are pushing your site to rank in a site like TechCrunch.
Types of performance testing
Performance testing includes load testing, stress testing, peak test and endurance testing. We provide a brief definition of each below:
Load testing definition
A load test is a type of software testing which is conducted to understand the behavior of the application under a specific expected load.
Stress testing definition
Stress testing means pushing a site beyond normal operational capacity, often to a breaking point. Stress testing is normally used to understand the upper limits of capacity of a system.
Peak test definition
The peak test is designed to analyze the system’s behavior when exposed to intensity peaks mixed with regular load, showing the recovery following the increased load. It’s used to check how the server copes during the busiest periods.
Endurance testing definition
Endurance testing involves testing a system with a significant load extended over a significant period of time, to discover how the system behaves under sustained use. For example, in software testing a system may behave exactly as expected when tested for an hour but when the same system is tested for 3 hours, problems such as memory leaks cause the system to fail or behave randomly. The goal is to discover how the system behaves under this kind of sustained use.
The worst examples of failures to perform load testing
Let's review some of the most notable cases of brands dropping the ball when it comes to performance testing.
The largest US department store chain, Macy’s, crashed under Black Friday heavy loads. The website and app were unavailable because of the huge amount of traffic it received on Black Friday, 2016.
Donald Trump’s presidential victory not only surprised the pundits but it also came as a shock for the Canadian immigration site. On the night of the US election result, Canada’s official immigration experienced a surge in traffic from anxious US citizens wanting to skip the border north. The website crashed under all the additional stress.
Star Wars: Rogue One
The much anticipated Star Wars: Rogue One movie launched a site where fans could buy pre-sale tickets but the website crashed within a few hours after selling started due to the large numbers of people attempting to buy tickets.
Old Navy’s landing page on Cyber monday: "Hmm, something's not right. We appreciate your patience, and we're working to fix the situation."
In June 2015, Southwest Airline’s website was down for 2 days because of overwhelming responses from customers to a sale that the company had launched. The business impact in lost revenue alone to Southwest Airlines was $5.64 million.
Performance testing is an indicator of how efficient entire website are. Different types of performance testing will prevent your website from delays and server overloads which lead to losses in terms of customers and profits.
Load testing helps to determine how many users a system can handle and different user scenarios to focus on the most critical parts of a website. It’s good practice to perform load testing as often as possible to ensure that your system is on point.
Stress testing helps to determine endurance capabilities of a system. Stress and peak testing should be performed when you’re expecting a greater amount of traffic or activity than normal.