There are two areas of advice commonly given that we believe need to be treated with caution. The first is to significantly invest in new technology because the current technology is not delivering returns.

We would advise any leader looking at this problem ensures that everything possible had been done to make the current technology work effectively before committing to another investment that is more than likely to deliver the same underperformance. Whilst it is true that platforms can come to the end of their useful life, it is equally true that many platforms fail to deliver because they are not made to work well. The second questionable advice is the use of random data points to prompt action, especially when much is being made of data correlations where there is little, if any, evidence of causation.

The dangers of data

There is a trend in some parts of business to be obsessed with data regardless of its utility. For example, in our retail work, whilst being 'in stock' and fulfilling deliveries is critical to sales success, we have seen little evidence that getting it to the customer more quickly is a defining competitive edge for the majority of customers. It is clearly important for a segment of the market but there is no robust evidence to suggest that this is considerably more the case now than it was 20 years ago. The right product at the right price point sold in the right way with clarity over shipping time wins nearly every time over any other combination.

We highlight this because it is a good example of what is now a standard consulting argument for multiple, layered and complex data reporting: Most direct-sales companies (retailers, banks, travel services), for example, measure the performance of their spending by analysing what consumers do just before making a purchase, eg googling a product. That level of analysis is too narrow because it doesn't take into account the advertising, social media chatter, and other media that also influence the consumer during his/her decision journey. In the world of on-demand marketing, marketers will need to develop a broader understanding of the complex series of interactions that make up the entire consumer decision journey.

This is a good example of selling services rather than value. There is no doubt that online channels can give you far more data and there are many products and tools out there that can cover endless data points. But, buying these arguments doesn't just keep consultancy square footage rentals going upwards in smart business districts; even more worryingly it can ensure that you drown your organisation in data and in doing so blind it to information that is critical to success. This is one of the biggest challenges that any eCommerce team faces and it is part of a wider problem that the big data promoters still need to resolve as they push ever harder to sell their services into business.

Tim Harford, one of the UK's leading statisticians, makes the point in an article in the Financial Times that the big data movement misses the importance of understanding causation and issues of potential sampling bias – just because the sample is big, doesn't necessarily make it representative.

There are some great case studies out there, be they Amazon, Tesco supermarkets in the UK or Target supermarkets in the United States, where stories abound about the power of their algorithms and their ability to make money through using these to predict purchase patterns from past behaviour. The truth behind these, however, seems to be much more straightforward – given the amount of data they hold on so many of their customers they have to get it right some of the time. In other words for every time they get the right voucher or recommendation, they fail far more times. In our (equally unrepresentative) experience Amazon just presents us with more of what we have just bought, whether or not we liked it or want to take our interest further and our own Tesco supermarket vouchers are rarely used.

As Harford reminds us: "None of this suggests that such data analysis is worthless: it may be highly profitable. Even a modest increase in the accuracy of targeted special offers would be a prize worth winning. But profitability should not be conflated with omniscience."

There will no doubt be a rush into big data activity in many businesses but to be profitable and to grow faster than your competitors does not necessarily require the data factories that are now being sold as the solution. There is an alternative to this approach that can provide a business with data that provides insight into purchasing behaviour. We call this deep data and define it as the combination of multiple data sources, qualitative and quantitative, that can generate informed insight about the customer in the market. Deep data still won't give you the answer, however; it gives you the insight to generate a hypothesis that you can test and through testing learn more about the customer in the market.

Your digital channel gives you both access to deep data and the opportunity to test and learn in real time very quickly and cheaply. If you have a fair amount of traffic then it can do this with a sample size of customers looking at that time for that product or service. This can give you considerable confidence in the result. If your test produces statistically significantly more sales than your current execution then if you change your execution to the test, sales will grow. To do this well doesn't need a lot of staff or incredibly costly technology. It needs commercial sense, an open mind and a very disciplined process. Getting this right should be at the centre of any eCommerce strategy.

Many organisations are facing a challenge of commercial credibility. It is difficult for leaders to identify the source of the problem as quite often consultancies and agencies are just as much a part of it as the capabilities of the in-house teams themselves. We are more and more convinced that this situation arises not because businesses employ incapable people, far from it, but because they are mis-organised.

They are using good people to do things that are not necessarily effective in ways that are inefficient or inappropriate. In any other discipline experienced senior leaders would spot this, although perhaps not at first. In a new discipline, made opaque to senior leaders through technology and jargon and often handed to an 'expert' to direct, it can be nearly impossible to spot. However, spotting it is essential if you are to break the cycle of underperformance.

The killer questions below are the best we have found that indicate a diagnosis of mis-organisation.

Killer questions:

Poor returns on investment

  • Are you experiencing low/no sales growth in the digital channel?
  • Have you seen repeated investments in the channel failing to reach promised outcomes?
  • Are you hearing channel performance being attributed to technology issues or competitive responses?

Constant change to sales execution

  • Are you seeing regular small changes made to website sales execution?
  • Are you seeing inconsistent customer engagement over time?
  • Are you being presented with re-designs and changes to presentation based on 'expert opinion' and not tested on real customers in real time?

Failure to engage the customer

  • Are there no opportunities for customers to feedback online?
  • Is your online engagement through a 'tick box' poll?
  • Are you seeing no cross-channel interaction where best offline sales executions are used online and vice versa?

Failure to test and learn

  • Is there no structured process that takes customer insight and tests potential responses to find the bestselling one?
  • Are you seeing the testing of solutions based on 'expert opinions' rather than solutions based on 'customer feedback'?
  • Are you seeing that testing is always of small differences rather than a mix of small and big alternatives?

Inflexible and slow IT change processes

  • Are you applying the same processes to change in the online channel as to change in major systems?
  • Is there a long list of improvements and changes awaiting execution?
  • Are you running change processes in a timescale that drives the prioritisation of the urgent over the important?
  • Are you applying complex and costly performance analysis and testing solutions?

If you are answering 'yes' to anything over 25% of the points above then you should worry; 50% and above and you should be very worried. Getting this right takes time and is a significant change challenge in its own right.

Next month we explore how your digital channel gives you the opportunity to test and learn in real time, quickly and cheaply.

This is the second extract from Leading Digital Strategy, a comprehensive book on the challenges of digital leadership, written by Professor Chris Bones and James Hammersley. Over the coming months, Essential Retail will be reflecting on the main themes from the publication, via a series of articles written by James.

Click here to read article one, 'Why are you so far behind in the digital revolution?