Naturally he refused
to play – but was persuaded to relent and to try his best with the
hastily-repaired instrument. The result: not a catastrophe, but a masterpiece.
The faults of the piano refocused Jarrett and forced him to explore new ways to
improvise. The concert recording,
The Koln Concert, is one of the most
popular jazz records in history.
This isn’t just a
matter of jazz genius: a few years ago, when the London Underground suffered a
partial shutdown for 48 hours because of a strike, Londoners re-routed their
commutes, sometimes using buses or overground trains. After the strike was
over, an analysis of the electronic turnstile showed that tens of thousands of
people had changed their route and then never changed back. The brief
disruption indicated they’d been doing the commute wrong all their lives.
I talk through these
examples, and much more in the latest episode of the Data Brilliant podcast
with Qlik’s Chief Data Officer, Joe DosSantos. When thinking about the above
examples of how disorder can actually lead to innovation, we cannot help but
draw parallels to the awful pandemic we’re all facing. Nothing will compensate
for the hundreds of thousands of lives lost – nor the devastation wreaked on
people’s livelihoods – but there is ample historical precedent for the old
saying that necessity is the mother of invention.
The Promise of Data
As Joe and I are
both data people at heart, we talked a lot about the promise of data. In
particular, we discussed how too often people are willing to shrug and to
dismiss statistical claims as inevitably distorted and misleading. That’s a
tragedy: in a complex world there are truths we can perceive with statistics
that have no other way of knowing. I’ve been championing the idea that
statistics are a vital tool for seeing details about the world that we
otherwise could not – like a telescope, a microscope or an X-ray. We simply
need to be curious and open-minded – a healthy scepticism about surprising
claims should not dissolve into knee-jerk cynicism.
While there are many
important technical skills for a data scientists or analyst, that spirit of
curiosity is as important as any of them. We need to be willing to look deeper,
and broader – and to ask what our categories mean. Inevitably, a statistical
view means drawing sharp distinctions where in reality there are shades of
grey. That’s okay – but we should be conscious of the fact that messy reality
does not always fit into the tidy boxes we create.
Learning From Mistakes
As we all know, it’s
hard to get things right first time, whether we’re talking about a forecast, a
statistical analysis, a business venture or a golf swing. What matters is the ability
to notice and correct your errors at the earliest possible opportunity. That’s
why Joe and I discussed Philip Tetlock, the psychologist and forecasting
expert, who writes approvingly of “foxes” and “actively open-minded thinkers” –
the point here being a willingness to test your ideas, seek alternative views,
and change your mind.
That’s good. But as
Joe pointed out during the episode there are other approaches – for example,
rapid iteration, or the use of A/B testing, or experimenting with prototypes.
What all of these approaches have in common is the central realization:
could be wrong.
We don’t like staring at that uncomfortable fact, but the
sooner we recognize it the easier it will be to learn from our own errors.
Tune in to Data
Brilliant now to listen to Tim and Joe’s discussion in full.