As data, analytics and AI become more embedded in the day-to-day operations at most organizations, it is clear that a radically different approach to data architecture is needed to create and grow the data and AI-centric enterprise.
In this blog post, I am going to take you on a journey through the Analytics Data Pipeline, exploring how it delivers against modern demands for data agility and informed actions to help organizations achieve Active Intelligence.
First of all, what is an Analytics Data Pipeline and how it is different from a traditional data pipeline?
An Analytics Data Pipeline is a framework with a set of architecture and tools that go beyond traditional data movement, automation and transformation.
Let me explain it by exploring the data journey end-to-end: from raw data to informed action.
Find it and Free it
The first part of the Analytics Data Pipeline enables a DataOps approach, vastly accelerating the discovery and availability of real-time, analytics-ready data to the cloud of choice by automating data streaming (CDC), refinement, cataloging, and publishing.
This first part “frees” the data from siloes, taking raw data (internal, external, and derivate data) from wherever it is housed (in the cloud, multi-cloud or on-premise) and continuously delivers it where it needs to be, reflecting changes in real-time.
This is where raw data becomes analytics ready.
Once the data is freed, this architecture applies data lineage, smart data profiling and cataloging to the data, so that it becomes analytics ready. This means users can easily find and consume data for consumption with trust. At IAS, for example, the Data Catalog acts as a secure, governed repository, from which users can organize and access analytics-ready data.
At the next stage in our journey through the Analytics Data Pipeline, analytics ready data becomes business ready. Users can begin to understand the data and apply timely business logic and context in a governed manner for insights generation.
By leveraging the power of Qlik’s unique associative calculation engine, users can apply - on the fly - new business transformation logic to analytics ready data.
With the governed transformation agility that Qlik’s associative engine provides, users can apply trusted data to decisions made in the business moment. During the disruption of the last year, for example, while most organizations had analytics ready data, they were not always able to get answers that empowered them to make timely decisions. Data was often not reflective of the business moment as conditions and dynamics changed rapidly around organizations.
Qlik’s customers, however, had a huge advantage. They were able to apply the new business rules to analytics data, making it business ready. At University Hospitals of Morecambe Bay NHS Foundation Trust, for example, having access to real-time data was critical to its management of coronavirus within its emergency department. With information readily accessible through Qlik Sense dashboards on the infection status of patients both in its care and in incoming ambulances, it was able to take immediate steps to best protect other patients and its staff. Its approach was so impactful, seven further NHS Trusts in England adopted the same approach.
In final leg of our journey, we see business ready data become actionable. This is the key differentiator of Active Intelligence and what intelligent analytics data pipelines empower.
The system embeds analytics into automated workflows, delivers sophisticated, context-aware alerts in real-time, and, as change happens, triggers automated business workflows. This stage enable intelligent systems to surface analytics-based signals and take governed actions much earlier, even before a dashboard is built.
I believe most organizations are facing an unsurmountable challenge in trying to drive informed actions with the traditional data pipelines and architectures. Fully automated or hybrid (with human in the loop) informed actions that can be taken by a system with the power of data is what will finally make the promise of BI and analytics a reality. And this Active Intelligence and it can only be achieved with a modern Analytics Data Pipeline.
If you want to learn more about the different components of the Analytics Data Pipeline and how it can help you achieve Active Intelligence, you can find more information here.