After ONS retail sales delay, what data do retailers rely on?

Observers of the retail industry will know that there is more data flying around than ever. The recent pause of the release of ONS retail sales statistics also highlights that a lot of that data could possibly come with caveats.

Alongside official data releases like retail sales and widely trusted third-party data providers like Worldpanel by Numerator and NIQ, new suppliers are continually creating innovative streams of data that retailers may benefit from paying attention to.

All that external information comes alongside retailers’ own internal data. It would be inaccurate to say that all major retailers have a full handle on the data they can possibly use that they generate themselves, let alone the data that exists beyond their walls.

New research from Retail Economics and wealth data analytics firm beBettor lays out that landscape clearly.

In short, there are two sources of data: internal and external. And very broadly, there are two types of data: structured and unstructured.

Structured data includes sources like pricing information and sales. This is information stored in a machine-readable format (like a spreadsheet) that can easily be pulled into analytics dashboards.

Unstructured data is the messier information that needs some data wizardry to pull into a useable state. Internally, that may include customer support interaction logs or store manager reports. Externally, you are looking at sources like social media posts, customer reviews and basically anything that exists about your brand or category area online.

That unstructured information is powerful: Think mining chat logs to see products that customers are continually running into difficulties with or analysing social media trends to help with product development. However, given the additional work required to make it useful, it is little surprise that unstructured data is utilised less often than the pre-tabulated information.

Based on surveys with retailers, the authors of the report gave a “utilisation score” of 56.4% to external, unstructured data by retailers. That compares to a score of 64.2% for internal structured data, which was deemed the most heavily utilised category of the four.

Internal unstructured data also lags with a score of 59.5%. In short, this all means retailers are not making full use of any data category, but that is particularly true of the unstructured information.

“External data is one of retail’s most undervalued assets. It offers a window into the financial and emotional reality of your customers beyond your shopfront – to anticipate trends, spot demand shifts and offer more relevant engagement,” said Retail Economics head of commercial content Nicholas Found.

“Retailers who understand this shift will be strong players in the new era of hyper-personalisation.”

Reliability of external data

The question of what data retailers can rely on has come up continually in the press in recent months and even this week. The Office for National Statistics announced on Tuesday that it was delaying the release of July retail sales figures by two weeks for “quality assurance” issues, vindicating many economists who have been raising questions about the data for months. 

Among those raising concerns was the BRC, where economist Harvir Dhillon has pointed out inconsistencies in ONS data and highlighted the pausing of important data sources. Commenting on the delay, Dhillon said: “The delay in the ONS retail sales release makes it harder to get a timely view of the health of the UK retail sector. While we welcome quality assurance, ongoing issues with data reliability and seasonal adjustment methods mean that the value of these figures for policymakers and industry is limited until confidence in the methodology is restored.”

Does this matter to retailers specifically? It does, but is perhaps a distraction from the datasets they may be able to use to help with operational questions.

According PwC partner and AI lead Oz Ozturk internal sales and loyalty data is far more important than high-level outputs like sales monitors.

“The high-level directional data is interesting for long-term strategic planning or investments,” Ozturk says, speaking before the news of the ONS data pause. However, he adds that “if you can curate your own data sources that explains your consumer, they are going to be more loyal because you are serving them better”.

How detailed external data is being used by retailers

Highly detailed external data meanwhile, both structured and unstructured, is proving incredibly useful to some data-advanced retailers particularly with steps forward in AI and demand forecasting models.

Amazon, for example, is pulling in time-bound data like weather patterns and holiday schedules to run through its models and combining that with historic sales data to improve its inventory forecasts. Walmart, meanwhile, recently announced that it was using software that calculates climate risk to help influence pricing decisions.

Not every retailer is yet at this point and may require support from a supplier to start working with more detailed data. One example is a company like Quid, which pores over masses of unstructured data from social media and the wider online conversation to help generate insights that can guide retailers’ planning and marketing decisions. 

“We have customers taking that feed and it runs into all sorts of parts of their business,” says senior vice president Ian Davis. That includes everything from merchandisers through to those running forecasting models, whether that is internal teams or external contractors.

Quid chief executive Anthony Lye adds that retailers are often generally looking for useable information though, rather than just a data feed. “The industry just doesn’t have an endless supply of data scientists and analysts sitting around, so what we have done as a company is go from selling software to selling insights.”

This sentiment is mirrored by Retail Economics’ Found, who says “the difference between interesting data and actionable insights is the ability to embed data into workflows and culture. This means linking it to specific commercial outcomes, such as greater personalisation, sharper pricing or improved loyalty, and ensuring it’s easily accessible to decision-makers across the business.

“The ability to pull this off effectively is often limited by in-house resource and knowhow. This is where partnerships matter. Retailers must open the door to third-parties to draw in new data, insights and capabilities. If they don’t, their competitors will.”