Three Essential Attributes to Modernize Your Data Real-Estate
Data Warehouse, Data Lake and Real-time Streaming
I’ve got a confession. I’m a data guy. There, I said it. In fact, I’m such a data guy that it borders on obsession. I’m always scanning the feeds for articles about data. Well, you can imagine my delight when I stumbled upon a recent blog post by McKinsey & Company’s Henning Soller and Asin Tavakoli that highlighted three requirements to transform your company into a data-driven powerhouse. I must admit, however, that I often read with skepticism, and it’s not long before my inner monologue starts dismissing the authors position with a large “Pah! What do they know!”
This time something was very different. The McKinsey crew were onto something that I could agree with. Some readers might say that it’s just confirmation bias, but I’d have to respectfully disagree. The reason is that the McKinsey authors were describing market forces I knew very well and advocating an approach we advise here at Qlik.
The three attributes for digital transformation recommended by the McKinsey authors start with a cloud data warehouse, add an open data lake and finish with a real-time data streaming. If I didn’t know better, I’d think they were paid to promote Qlik’s thinking. For the record, they aren’t. So, now you can see why I was so stunned. There it was in black and white, with a reference architecture to boot.
McKinsey’s reference data architecture is based on three pillars that sit on a foundational data-ingestion layer:
- The data warehouse pillar supports predictable, highly critical reporting, such as regulatory compliance and financial reporting. In my last blog post, I commented how many of these great options today exist in the cloud.
- The data lake pillar is ideal for less stringent reporting needs, as well as advanced analytics use cases that require large-scale data processing.
- Real-time streaming. This pillar enables real-time use cases as well as rule-based analytics. The transactional databases that serve the pillars are connected either directly (streaming) or through the data lake (exhibit).
Now, let’s compare the former diagram to the architecture of Qlik Data Integration that you can find on Qlik.com.
The similarities are striking, while the differences are mainly in layout. McKinsey’s diagram operates in both the horizontal and vertical; the Qlik diagram reads linearly from right to left. Both contain the same conceptual components. In an event, the architectural benefits are clear.
Companies who base their data architectures on these core principles can be more agile, scalable and resilient. They can accommodate additional use cases for the data and prove more cost effective. In fact, I’ll point you to a recent Nucleus ROI report that documented how one company discovered a 400% productivity improvement for its cloud data warehouse. You can download that report here.
Conclusion
Business leaders around the world recognize the value of becoming
a data-driven organization, and many companies have begun to implement advanced
analytics and artificial intelligence use cases. However, they often struggle
to morph their existing legacy systems to support new requirements and analytics
at scale. This was highlighted in a recent McKinsey and Company article, which
stated to become a data-driven organization you must modernize the IT estate
with a cloud data warehouse, an open data lake and a real-time streaming platform. Not
surprisingly, these three tenets form the backbone of Qlik’s Data Integration platform.
