Machine-learning models trained on pre-COVID data are now completely out of whack, says Gartner

That AI-powered product and price recommendation engine? Useless now

Machine learning models built for doing business prior to the COVID-19 pandemic will no longer be valid as economies emerge from lockdowns, presenting companies with new challenges in machine learning and enterprise data management, according to Gartner.

The research group has reported that "the extreme disruption in the aftermath of COVID-19... has invalidated many models that are based on historical data."

Organisations commonly using machine learning for product recommendation engines or next-best-offer, for example, will have to rethink their approach. They need to broaden their machine learning techniques as there is not enough post-COVID-19 data to retrain supervised machine learning models.

Advanced modelling techniques can help

In any case the 'new normal' is still emerging, making the validity of prediction models a challenge, said Rita Sallam, distinguished research vice president at Gartner.

"It's a lot harder to just say those models based on typical data that happened prior to the COVID-19 outbreak, or even data that happened during the pandemic, will be valid. Essentially what we're seeing is [a] complete shift in many ways in customer expectations, in their buying patterns. Old processing, products, customer needs and wants, and even business models are being replaced. Organisations have to replace them at a pace that is just unprecedented," she said.

"Models that are based on extensive historical data; any sort of planning based on past performance and, even some models about customer behaviour [will no longer be valid] because things have changed significantly, and as a result, customers are behaving very differently," she said.

Supervised machine learning would need to be embellished with alternative techniques to allow organisations to adapt to new economic behaviour more rapidly, she said. These include reinforcement learning, which trains a model to make a sequence of decisions in an uncertain, potentially complex environment. It also includes distributed learning techniques based on multi-node machine learning algorithms.

The problem is few organisations have the skills and data infrastructure to put these alternative approaches into practice at the moment, Sallam said. "We were suggesting these techniques will be mainstream three to five years from now; the global pandemic will likely accelerate organisations looking into and building skill sets and capabilities," she said.

Organisations looking to accelerate their application of machine learning will improve their data management and pipelines, Gartner said in the report Top 10 Trends in Data and Analytics, 2020.

For example, the analyst estimates that by 2023, teams deploying active metadata, machine learning and data fabrics to connect, optimise and automate data management processes will reduce time to integrated data delivery by 30 per cent. Similarly, augmented data management will reduce the reliance on IT specialists for repetitive and low impact data management tasks thereby freeing up to 20 per cent of their productive time.

Outside the most advanced business in, say, banking or e-commerce, most organisations are just starting to operationalise their machine learning efforts. To say that one of the most standard, established approaches is no longer valid because data from the past will no longer be a good predictor of the future is a big call from Gartner specifically, as you might imagine. It calls for greater investment in new roles and capabilities. Whether cash-strapped businesses are in a position to take action on this is another matter. ®