Dashboards, Models and Mud
When it comes to getting the most from your data, sometimes you have to be willing to get a little dirty.
I recently discovered Mission Control: A History of the Urban Dashboard, by Shannon Mattern. It's a fascinating article on the use and limitations of data and dashboards in the context of urban planning and management.
Now before I hear you shout "sacrilege!", let's take a look at what's behind the idea of the dashboard.
The term "dashboard" originates from the name given to a protective barrier at the front of a horse-drawn carriage, a board which protects the rider from getting covered in mud and filth (and any of the horse’s biological outputs). When this was adapted for the automobile, it protected the driver from the engine (and the output from the mechanics). The modern data dashboard protects the viewer from the mess of the raw data. It's a way to handle the problem of signal and noise and information overload, thus helping people focus on the information that matters most. Now as Mattern points out, many complex systems such as cities simply can't be boiled down to a few pertinent KPIs, no matter how much we'd like that to be the case. The KPI dashboard ultimately restricts our view, giving us blinkers (to continue the horse and cart metaphor). This can be useful, but it can also lead to a kind of tunnel vision. As James Richardson loves to point out, KPI can very easily come to stand for “Killing Personal Initiative.”
Lately, I've been thinking a lot about that idea, especially in relation to automation and computer models. I've been experimenting with a few computer vision APIs and their use for seeing and describing the content of images. It's incredible what they can achieve, even the publicly available ones are pretty impressive. It is especially impressive compared to four or five years ago when we first talked about natural analytics and the advantages of using our human skills for processing visual information.
But most of these models are still blinkered, in the same way that the viewer of the dashboard can be. The ones that work best are either extremely broad (high level, generalized categories as above – ‘a bicycle parked on field’, rather than ‘a blue Swedish Pilen bicycle, with black mudguards, standing on a daisy strewn grass lawn, near a marina in front of a cement works) or super narrow (focused on a very specific attribute, such as license plate reading). When the subject of the image is beyond their frame of reference they often struggle. At best, those blinkers can result in some humorous mistakes when things that aren't part of the model's realm of understanding come into view.
At worse it results in false positives which can lead to bias, prejudice and discrimination, and when that's reflected back from the model at scale, it can be very ugly, as Google found out a few years back.
The thing is, we often need a lot more than a simple score, closed set, or single perspective when trying to understand the information at hand. We need a variety of views or layers, each enabling us to frame the previous, to give it context and shed light on any misreading or misunderstanding. Mattern gives the example of Patrick Geddes' The Outlook Tower in Edinburgh from the early 1900s. The tower enabled people to understand the place (Edinburgh), by examining the 'live' view, both in macro and micro detail (through a camera obscura). Then it enabled them to position that information in an ever larger context (city, country, language, continent and world) by walking down floor by floor through the tower’s exhibition. This idea of framing and then reframing the observed or reported data is a fundamental part of research and analysis. It's why for many years people have created Qlik apps using the idea of a stepped analysis, from dashboard (indication), to detail (analysis and association), to report (sharing insight and evidence). That 'detail' step is essential as it adds context to the data, but it's not always enough. Sometimes you need to get down in the mud and the filth to truly understand what your data is and isn't saying, or what you can and can't see. Not only that, but you may need to bring new perspectives to the mix, so that you can reframe and re-contextualize what it's telling you, in the same way that Geddes reframed the data from the view of the city using country or language. Those additional perspectives may even come in the form of qualitative data, and all its personal, physical, and human messiness.
Dashboards and models help us process and synthesize the overwhelming mass of data we have at our fingertips. But remember, you are always getting a reduced view of the world and sometimes you'll need to jump down into the mud and get dirty, as that’s where the real story is often hiding.
For those who like an even longer read, here’s a link to Shannon Mattern’s Mission Control: A History of the Urban Dashboard
Image by National Library of Scotland [No restrictions], via Wikimedia Commons