Test or prepare to fall behind

Despite being critical to any commercial endeavour, there is an inherent problem in giving people choice. There is always the risk that people choose something other than what the person offering the choice had hoped.

If you own or run an eCommerce site, you do have to offer each visitor a decent number of choices in order to deliver an exceptional shopping experience that compels the visitor to take specific actions. In the simplest sense, this also needs to help drive commercial performance.

Regardless of whether the KPIs are enhancing existing customer relationships or acquiring new customers: the better the shopping experience, the more likely visitors will do what you want them to. So how do you offer choices that create customer satisfaction whilst still delivering on commercial success?

A/B test, and test again

Discovering the best route to offer customers is the realm of A/B testing. It’s the practice of delivering two different experiences to different segments of visitors at the same time and comparing which variant drives more conversions.

This needs to be embedded in a process for increasing the number of visitors to a website who take a particular desired action. A/B testing is also the clearest demonstration of measuring what visitors want. There are, however, a set of strategic considerations that need to be made before offering these choices.

Firstly, retailers must rid themselves of the idea that they think they know what the customer wants.

The best way to do this is asking and observing customers, first-hand. Online surveys, feedback systems, card-sorting exercises, and usability testing groups all yield priceless insight to refine the choices a business offers its customers.

At this point, the business can then develop a sound hypothesis that acts as a starting point for further investigation. This is also the best time to define what a successful set of choices looks like and to define why the retailer is running the test in the first place. The more precise the hypothesis, the more valuable the result.


For any test, many variables mean that an experiment takes longer and of course, the more variables, the greater interaction between them. So, the less clarity can be realised on what exactly causes a given event. A/B results will often fluctuate in the shorter terms so it's important to persevere and ensure that the test has a good chance of reaching statistical significance. This is not just about the ratio of sample size and the number of conversions.

Businesses must measure fair comparisons between the control and variation so that the results are not unfairly skewed. This critical insight is to identify exactly which change caused the test to succeed or fail – that is to drive the shopper to make the desired choice. This must be fully documented so that a business can refer to it in future and build further hypotheses for follow up test.

Assessment and action

If a retailer does not experiment to ensure it is in tune with an ever-changing online audience, it will stop evolving and fall behind the competition who have learned from experimentation culture and optimised shopper journey. For instance, cart abandonment can be reduced by allowing shoppers to check out as a guest, decision fatigue could be reduced by opting for a horizontal navigation. The value of A/B testing is that it shows how a business can stay relevant not only from user experience (UX) point of view, but also from the view of search results returned during their shopping journey; if a misspelled search returns no relevant results, an impatient shopper may leave the website. The key is to identify opportunities, construct hypothesis to optimise, experiment and implement once a clear winner is identified in a statistically significant manner. Even across multiple variables, it is very easy to analyse based on the results observed. It is usually possible to determine a clear “winner” and a “loser”.

This can help identify which variations create the highest levels of engagement. Based on the initial hypothesis under test, there are different metrics for determining what qualifies as engagement such as click-through rate, average order value, customer lifetime value to name a few.

A/B testing also adds a significant element of objectivity to these decisions. The assessment of shopper behaviour reduces risk and increases the likelihood of success. When implemented in a series of strategies, the result is a more efficient use of resources and improved outcomes.

One early benefit is usually reduced bounce rates as the data shows why visitors leave a site. The business can then take steps to make the experience more compelling and relevant. Likewise, there should be less cart abandonment once a business has identified strategies to improve checkout completion.

A/B testing can also be beneficial for testing different ‘What if?’ scenarios across all areas of an eCommerce site. It can guide decisions on landing pages, listing pages, product detail pages, adds to basket and even purchases.

Throughout the entire eCommerce experience, having a solid A/B testing strategy and tools in place enables businesses to realise what choices they need to offer their customers in order to drive engagement and ultimately, sales.

In short, it’s an easy decision to move forward with.

What’s Hot on Essential Retail?