By Zank Bennett, Bennet Data Science.
COVID-19 is all over the news.
And rightfully so.
It’s a tremendously important topic.
And it’s having massive effects on the accuracy of AI models.
Before we dive into how AI has changed, let us first say from all of us here at Bennett Data Science that we sincerely hope that you are well and getting through the hardships as best you can.
Businesses are undergoing tremendous change.
It’s hard to find an industry that isn’t drastically affected by COVID-19.
In attempts to keep up with either precipitous constriction or massive immediate scaling, companies are seeing their tried and tested methods of interacting with their customers rendered almost meaningless.
That means marketing personalization, lifetime value calculations, product recommendations, and churn models, are all inaccurate at best and misleading at worst.
AI uses past actions to predict future events.
And nothing in the (recent) past is anything like what we’re experiencing today.
So AI is confused.
It’s as though we’re all suddenly speaking a new language to Alexa or Siri or Google Assistant.
Of course they wouldn’t “understand”.
They’d be useless.
Global purchase behavior is now speaking a different language.
And AI doesn’t understand.
Companies are scaling the only way they know how, by putting workers (people) in the trenches to make decisions about how to support rapidly changing customer demand.
And they’re doing it largely without the AI they’re used to relying on because the models are not accurate anymore.
We’re seeing companies going back to making business decisions based on recent data from dashboards in place of scalable, automated AI.
And for good reason.
In many cases, AI is useless because it’s never seen anything like COVID-19.
This often means that teams are overwhelmed by all the data to process, sift through, and make actionable.
Because it’s not enough to “discover” an insight.
Companies need to act on the insightful information.
And machines can do that at scale, company-wide.
Humans cannot.
So, things back up.
While this sounds grim, there’s a lot that companies can do to keep their intelligent systems working for them and their customers in this ever-changing environment.
But before we get to this, it’s important to understand the specific hurdles they encounter.
I’ve had several conversations over the past few weeks to discuss how various businesses are fairing, and how AI is working for them, given the current state of the world.
The majority of executives and data scientists I talked to pointed towards the importance of spending more time looking at their data to understand how recent developments changed their business.
Data scientist Oleg Żero looked at it holistically, “The upcoming corona-induced crisis will do the same as any other crisis did.
It will sink the heavy and stiff and propel the flexible.
From a higher level, reactions to what’s going on are all over the map.
”To his point, some of our clients are doing ok, having had distributed teams from the beginning, but comfort levels vary widely from industry to industry, and many companies are struggling to understand what the future will bring.
E-commerce and delivery services, on the other hand, are set to thrive in the short-term.
Amazon alone is looking to hire 100k employees.
But for every business thriving, there are many more in dire condition.
Either way, everything has changed, and companies need to adapt.
Reza Sohrabi from StitchFix points this out as well, “I dont think there are any businesses that are not impacted by the current pandemic.
So, I believe companies have to adapt to the new situation and change gears as to how they do data science.
”One of the big challenges we’re seeing and hearing is that AI-generated predictions are unable to forecast future events, given current market and customer behavior fluctuations.
Predictive models rely on historical data to predict future events.
And when this historical data of customer behavior is nothing like what those customers are doing now, we have a problem.
Predictions go to rubbish.
Alan Murray from Fortune explains it well: “… modern-day ‘prediction machines’ are often based on data drawn from past behavior.
They aren’t prepared to deal with massive shifts in behavior—for instance when people inexplicably start hoarding toilet paper.
”Many different industries are currently impacted by changing data such as extremely long supermarket lines, erratic shopping behavior, and spikes in traffic to sites like public health and financial centers, reporting what appear to be denial-of-service attacks.
Companies are seeing signals that, just a few weeks ago, would have been anomalies or outliers.
These are the new normal now.
And most AI is ill-equipped to deal with them.
There is, however, something that can be done.
When we think about how to help companies in difficult times, we come back to the fundamentals: companies must engage their customers with products they’ll love.
It’s more important now than ever to understand what your customers really want and give it to them when they want it.
It’s also a great time to look at how organizations work with data and utilize analytics.
Eric King summarizes it nicely, “Theres not a better time to pull the lens back and look more organizationally at how companies are going to compete in the long run, as opposed to the disruption thats occurring this quarter.
” – Eric A.
King, President and Founder for The Modeling AgencyAfter all, the goal of effective analytics is to continually progress in the maintenance of customer lifetime value and all of its drivers.
These drivers of lifetime value include:These are important fundamentals to get right.
But it’s early 2020, and customer preferences around the world are quite literally all over the map.
If you have a business where personalization matters, there are a few changes that will help you retain value from data science:If you do these four things, your company will be far out in front of its competitors that may not quickly adapt to change.
“Companies need to be even more mindful of things like data freshness, distribution shift, and prediction calibration when running machine learning models in production.
With fast-shifting user behavior due to the effects of COVID-19, models are likely to rot faster and make bad predictions on unexpected user behavior.
” – Eugen Hotaj “[Recommender algorithms that consider] the time of the year, don’t make sense at the moment.
For example, the festival season is coming soon; normally, it would be reasonable to stock up on tents and recommend them to the youth.
It would be unfortunate to do that now.
Weights assigned to past actions must be reduced in a way that recommendations would rely mainly on trends of the last few weeks.
As a buyer, what I see now that in some cases, recommender systems are partly shut down.
” – Dániel Barta “As much as a model can predict specifics, it is important, in my opinion, to look at the underlying reasons.
” – Joseph “Goose” Aranez Bio: Zank Bennett is the CEO and founder of Bennett Data Science.
Related: var disqus_shortname = kdnuggets; (function() { var dsq = document.
createElement(script); dsq.
type = text/javascript; dsq.
async = true; dsq.
src = https://kdnuggets.
disqus.
com/embed.
js; (document.
getElementsByTagName(head)[0] || document.
getElementsByTagName(body)[0]).
appendChild(dsq); })();.