I’ve often commented that the current data market is in a constant state of flux, continually ebbing and flowing with the latest innovation and trend. Indeed, my first official blog post as a Qlikkie described how our data integration platform supported the Microsoft Synapse launch. That article was swiftly followed by a piece that discussed the resurgence of data warehouses. Shortly, thereafter, I penned two posts that focused on Snowflake – the built-for-the-cloud data warehouse. One covered practical advice for data practitioners, while the other described our companies' expanding relationship through the Snowflake Partner Connect Program.
Not to be outdone, my colleague Ritu Jain recently posted how Qlik’s data integration platform supports Databrick’s vision of a “data lake house” — a concept that combines the best approaches of data warehousing and data lakes for a single source of truth.
Those posts are proof that the data market is incredibly dynamic.
Questions, Questions, Questions
With all this change, it’s only natural to feel a little overwhelmed and uncertain about implementing a data strategy that can stand the test of time and reliably deliver data for analytic insights. However, the decision that seems to pay dividends, regardless of data requirement, is to implement a data warehouse. The data warehouse has several benefits. It can hold your entire data history, regardless of the retention policies of your transactional systems. It can also relieve the real-time query burden from transactional systems, too, by being the authoritative source for your historical and analytic reporting. Moreover, new cost-effective cloud offerings ensure that the data warehouse is now within reach of practically every business. Consequently, the question I’m most frequently asked is “How do I begin architecting a data warehouse?”
In answering, I like to focus on four major architectural questions outlined below and my responses to each.
The architectural considerations with Microsoft Azure
Synapse for a data warehouse are depicted as follows:
I realize my answers may gloss over some logistics. I also assumed
that you had chosen a data warehouse, assembled a team and performed some
initial requirements gathering. Despite those assumptions, building, filling,
consuming and changing a data warehouse remain the core competencies of
operating a data warehouse.
Answers, Answers, Answers
You might be thinking “Where can I find out more about architecting a data warehouse for the long term?” You’re in luck: We’ve partnered with TDWI and Microsoft to discuss the topic at a webinar, to be held March 11, titled “2020 and Beyond: Architecting Your Data Warehouse for the New Decade.”
Here’s a sneak peak of the webinar’s format and content:
David Stodder, Senior Director of Research at TDWI, will kick off proceedings. He’s going to discuss the current data market trends and the data requirements he’s hearing from his clients.
Matt Basile, Azure Data Program Manager at Microsoft, will follow David and will describe the latest innovations in Azure Synapse Analytics. Incidentally, Matt and I co-authored “Data Warehouse Automation in Azure for Dummies.”
I follow Matt and close out the discussion with the responses to the architectural considerations depicted in the above graphic and implied by my earlier questions. The webinar promises to be a very lively and informative session.
Register today to find out if your data architecture is ready for the challenges of a new decade and new era of data transformation.