Data as an Ecosystem

This reminded me of a conversation I had with Donald Farmer a few months back about analytical systems. Framing BI and analytics as a system is very useful, as it helps us understand and learn more about the usefulness of an artifact and more importantly it opens up the interactions beyond a user and a tool or a reader and a report. For me, BI and analytics are the most effective for organizations when viewed as a system, and not simply as a series of artifacts and tools.

An important difference is that individuals use tools but people participate in systems, and they aren't the only participants. Modern information systems contain a whole host of characters (people with differing roles, skills or intentions), digital services, bots and intelligent agents, and automated activities. The diversity and sophistication of these non-human participants is set to grow astronomically over the coming years. And it's the exchanges and learning happening between all these participants that increases the value of the system, augmenting both the human and machine intelligence with in it.

When we talk about BI and analytics we often talk about its adoption in an organization as a way to access its value and impact. We talk about how many people are engaged with their data and actively using it for decision making. This is usually measured based on their access and use of analytic tools and artifacts like dashboards and reports. Traditional BI tools have a very simple system: a small group of data experts use tools to create reports for a slightly larger group of people to consume. It's a one-way exchange that means if new 'questions' need to be asked then a new run through of the process is required and a new artifact created. That's the technological/business process view of the system. In reality the system is more complex with far more exchanges between the individuals, more discovery and feedback, usually mediated by the data experts and BI architects. Also, there's far more going on outside of that artifact creation process. Unfortunately in many of the analytic tools people are given to use these exchanges are rarely considered and the system gets reduced to this linear flow. This is usually because the focus is on end points and artifacts, neglecting the exchanges between people and the tacit knowledge they hold. In part this is why we often hear about how only 25% of an organization is actively engaged with analytics. Giving people access and more user friendly or even more sophisticated analytical tools has helped but it doesn't seem to have helped shift that number very much. A self-service analytics approach based on "if you build it they will come" simply isn't enough if it's still a tool and artifact model. It needs to be a BI system that services the diversity of the skills and intentions of the people in the organization.

As the law of requisite variety (according to Dan Lockton) suggests; in order to deal properly with the diversity of problems the world throws at you, you need to have a repertoire of responses which is (at least) as nuanced as the problems you face.

One tool, one artifact type, one size fits all thinking simply can't reach all the people in an organization. The key to engaging more people with their data and helping them grow both their confidence with analytics and their data literacy is in servicing them in ways that fit their behaviors and needs. For some it will be a static report, for others free form data exploration or drag and drop dashboarding, and for others perhaps it's a BI bot or intelligent virtual assistant. But all this (and more) should form part of a rich analytic system that has learning (for machines and people) and governance at its core - creating an informed, learning and data literate decision making ecosystem.