We missed it, again. But, could we have seen it, even with advancements in data analytics? Many people have referred to COVID-19 as an unexpected black swan event, after Nassim Taleb's famous book, which I’m rereadinging as I write this post. A black swan is an unpredictable event that is beyond what is normally expected of a situation and has potentially severe consequences.
But, according to Taleb himself, COVID-19 wasn’t a black swan. Like Bill Gates, Laurie Garrett and infectious disease experts, he predicted it, and even warned about it. According to Forrester, which reviewed the Fortune 50s’ current 10-K reports, half mentioned a pandemic in their risk profiles but only one (Apple) had insured against such a downside [1].
A few months ago, I saw posts on LinkedIn that the COVID-19 crisis should lead to a surge in advanced analytics. I responded to this, which created a fun discussion. My viewpoint is that, if advanced analytics struggled to pick up glaring red flags in the financial crisis, and now, also, with COVID-19, can we do something different? Do we need to up-level analytic methods? Whether it is a black swan or not, I think the answer is yes.
Too many models, relying on predictive analytics, struggle to pick up anomalies. These predictive models typically don’t work well when a critical input data point has never occurred in the past. Algorithms need good backward-looking data – and sometimes lots of it – to build a comprehensive model of the future. I recently had a conversation with someone who works as a data scientist, and he confirmed to me that removing outliers is the first thing people do. Models are often built for calm markets, placing a lot of value on backwards-facing historical data. When data scientists have to answer to their respective bosses, they can then say, “I backed this up with statistics and data.” Their reputation is on the line, and too often they select the path previously trodden, shying away from data that is less statistically safe.
Removing outliers has caused big mistakes in the past. A famous and somewhat contested case is that of the non-detection of the hole in the ozone layer by NASA. Outliers had been automatically filtered out from the data obtained from the Nimbus-7 satellite. It was only when the British Antarctic Survey observed the phenomenon in the mid-1980s that scientists went back and found the problem could have been detected a decade earlier if automated outlier filtering had not been implemented by NASA.
James Fisher recently wrote about a more recent example. In England, where the pandemic prevented students from sitting for exams, officials used AI to “determine” scores. The algorithm combined student rank with data from each school’s performance in previous years, and the outcomes, therefore, locked existing trends into place – and locked out outliers.
So, it’s clear that this is about much more than analytic techniques. It’s about a cultural change in how we see risk. It’s also about much tighter analytic governance. It’s also clear that we need to up-level analytic approaches to include a focus on outliers.
This could be done in multiple ways. Simulations introducing “unexpected” inputs don’t generally predict the future, but they can reveal how a system will react when the unexpected happens. What-if analysis presents options to us, upon which we can build contingency plans. According to Gartner, this area is becoming resurgent [2]. AI techniques and algorithms can increasingly surface “unknown unknowns” in Rumsfeld speak – i.e., anomalies outside preconceived notions and hypotheses. This can then be evaluated by humans. And, we need to embed advanced analytics in more places for quicker impact and immediate action.
So, what about the next crisis? In an interconnected world, we will, with a high degree of certainty, see more anomalies and more disruptions. That is the “new normal,” and the ability to prepare and act needs to be built into our corporate DNA. This means identifying and acting on signals and raising the importance of scenarios and options, with alternative hypotheses supporting the main scenario. And, in a chaotic world, use fresher data, where anomalies trigger both strategic and operational investigation. And, more than anything, it means active – as opposed to passive – intelligence.
References
[1] Forrester. Business Resiliency Will Become A Competitive Advantage.
[2] Gartner. What If More People Could Ask “What If?” — Providing Tools to Support Scenario Thinking.
