Blog: Measurement can make sense of the vast tsunami of online data

No comments

Management consultant and author Peter Drucker said “What gets measured, gets managed”.

Measurement is critical, yet knowing what to measure is significant in the complex, interconnected world of the online retailer.

The data comes in vast quantities and is often incomplete and lives in silos. Measurement is essential to understanding what’s happened and which part of the business needs focus – it requires turning the vast tsunami of raw online data into something meaningful, presenting it in a dashboard that makes it easy to interpret.

However, there are common pitfalls that retailers could face when measuring the data:

Common pitfall #1: Measuring outcomes

In physical retail, simple outcomes tell the story, metrics are mature and benchmarks are plentiful and meaningful.

Yet, with online retail it is a different story - outcomes online give only the illusion of information, and metrics like “conversion rate” are bandied around but the reality is that they are devoid of information. Why?

a) Conversion gives an incomplete picture. Conversion rate is but one component of the online profit tree. At the very least, conversion rates always need to be considered in the context of visits and profit per order. Conversion going down may not be a bad thing.

b) Conversion is an outcome of having the right products, pricing, availability, site experience and service. You cannot improve conversion “in one go”, but you can improve the individual and interconnected components: new/exclusive products, better availability, improved checkout flow, quicker delivery, and better service for VIP customers.

c) Conversion is naturally volatile. It’s a function of the source, destination and quality of traffic so the conversion rate online is always unstable, driven by the mix of traffic. Conversion must be systematically de-averaged to distinguish between benign fluctuations of mix, and more interesting absolute trends.

Common pitfall #2: Using the wrong inputs

To make sense of the online world you need to create a hierarchy of metrics – outcomes that tell you what’s happened at a high level; inputs that tell you what’s been done, and where to focus. However, it’s not quite that easy – there are three types of inputs to avoid:

a) Meaningless metrics whose behaviour is not necessarily good or bad. A good example is the “percentage of orders in a week from repeat customers”. If this increases, it could be either a good thing or a bad thing; it depends. It’s easy to increase it by acquiring fewer new customers.

b) Overly aggregated metrics where you end up with an average of an average. A good example of this is quoting a “Google cost per order” – the challenge here is that (i) averaging trademark and non-trademark keywords obfuscates performance, (ii) it gives no sense of margin/profit/customer value, (iii) it gives no sense of volume.

c) Self-congratulatory metrics which are akin to “marking your own homework”. An eCommerce operation whose primary metric is % of orders dispatched on-time and in-full (OTIF) may often give a false sense of success. The challenge is definitional – exactly how “on-time” is defined. Typically it is the company’s definition of on-time, not the customer’s.

The online challenge is to create a metric hierarchy that connects target outcomes [revenue/profit] with controllable inputs [things that people can affect]. It is the only way to make sense of what’s happened, and help inform where to focus.

The solution is to measure input metrics across the five trading elements; customers, site, marketing, product and operations, using a decision intelligent commerce tool that recommends options and exposes the value of each decision.