I have been having quite a bit of fun in this blog series exploring the “sliding doors” (or divergent paths) that organizations can take with their AI and data initiatives. The right door can lead to great value, as I covered in my previous posts on ESG reporting, data products, and responsible AI.
And yet that right door can prove quite elusive, even for major, big-name corporations that have made significant investments. We certainly saw this last year during the meteoric rise of generative AI: in a haste to jump on the train, steps were skipped, mistakes made, and ultimately value lost. It begs the question not only for the organizations that went down that path, but for all companies on the AI journey: do you know what makes an enterprise-grade AI strategy?
To help provide answers and guidance on this, I spoke with two company founders who joined Qlik a few months ago through acquisitions: Ryan Welsh from Kyndi, and Sharad Kumar from Mozaic Data. As seasoned entrepreneurs and technology innovators in the field of AI and data, Ryan and Sharad now lead new strategic product development initiatives for our analytics and data business units, respectively.
Wrong Door
“From my experience having built generative solutions for several years now, and delivered them to enterprises, the one thing I have learned is that Large Language Models as a stand-alone technology are not an enterprise-grade GenAI solution,” Ryan told me. “The main problem with LLMs is that they hallucinate, and they are stale – meaning they are not up to date with the most recent knowledge in the world, or within an organization.”
Here is an illustrative example of this. In April 2023, if you had asked ChatGPT “what is the gross margin of Apple for the last quarter”, it would have responded “Apple’s revenue was $95B and its cost of goods was $25B, resulting in a gross margin of $63B last quarter.” Sounds impressive except it’s completely wrong, bad math included. If you asked the same question today, ChatGPT would respond “I can't provide real-time data, but as of my last update, Apple's gross margin tends to fluctuate from quarter to quarter based on various factors such as product mix, sales volume, and production costs.” No wrong information this time, but hardly helpful.
Right Door
So, what’s missing? “An enterprise-grade GenAI solution is LLM + contextual enterprise data,” explains Ryan. “The solution is to feed enterprise data into LLMs: something called ‘in-context learning’ or ‘Retrieval Augmented Generation (RAG)’”. “RAG is one of the most effective techniques to provide data to LLMs,” adds Sharad. “As the name suggests - contextual data related to user prompt (question) is ‘retrieved’ and then the user prompt is ‘augmented’ with this data and passed to the LLM to generate an answer.”
To help illustrate this, let’s go back to our previous ChatGPT example. Here, in addition to asking a question, we also provide additional information to the LLM, which leads to the LLM answering the question correctly. Our prompt: “What is the gross margin of Apple for the last quarter? Here is the relevant financial data from the last 4 quarters…” (i.e. instead of just asking a question, we augment the prompt with contextual information) ChatGPT: “… the gross margin for Q4 2023 was approximately 45.87%.” Much better.
“LLMs need an effective data platform that spans the entire data value chain to deliver benefit to an enterprise,” says Ryan. “That’s why my team and I were excited to join Qlik a few months ago. For thirty years, Qlik has always brought the right contextual enterprise data to a user to answer a business question. Now, through the technology we developed at Kyndi, we can bring the right contextual data to an LLM for an LLM to answer a business question for a user. It’s why I truly believe that Qlik is one of only a handful of companies best positioned to lead the enterprise GenAI market, with a data platform that cuts across ALL enterprise data.”
Sharad adds: “I could not agree more. Foundation models like LLMs don’t understand company-specific data. They can’t answer questions about your business and hence we must first ground these LLMs with your enterprise data – but not just any enterprise data. I believe organizations should apply six principles to ensure that data is trusted and ready for AI:
Data should be diverse (coming from a wide range of resources) to remove bias in AI systems
Data should be timely to make accurate and informed predictions
Data should be accurate to ensure reliability and trustworthiness in AI
Data should be secure to safeguard the reputation of your AI
Data should be discoverable to enable use of relevant and contextual data
Data should be consumable to get your data ready for ML training and LLM integration”
The best way to ensure these principles are applied and effectively provide contextual data to your LLMs? Leverage data products. “Data products provide contextual and domain-oriented data in different ways for consumption, “ explains Sharad. “They contain rich metadata (business, technical and domain oriented) and highly curated datasets (files, tables, or vectorized) for consumption. Hence data products become the perfect vehicle for enabling RAG based applications. Building a RAG application then starts with first searching the right data product to use through the data product catalog / marketplace based on the question being posed. For example, a question related to sales may be best served by an “Opportunity” data product that is sourced from the CRM system based on the detailed metadata associated with this data product. Or a question related to an order may be best served by an “Sales Order” data product that is sourced from SAP based on the detailed metadata associated with this data product. Once the data product has been identified, the RAG application can directly query the data product dataset – tabular or vectorized based on the use case.”
Learn More at Qlik Connect!
You can expect to learn much more on this at Qlik Connect on June 3-5 in Orlando across our keynotes, breakout program and product showcases. In particular:
Ryan will present the “Unlock Insights in Unstructured Data with Generative AI” where he will review Qlik’s new offering based on the “answer engine” technology he built at Kyndi. In this session you will learn more about the benefits of a solution that will plug into your existing unstructured data stores and start asking questions – saving you the time and expense of having an internal ML team build it for you.
Sharad will present the session “Data Product Revolution: Transforming Data into Gold” where he will review what data products are, what challenges they help solve, how data products can provide contextual data to LLMs, and the best practices for implementing data products in partnership with Qlik.
Before Qlik Connect, please also be sure to join us on May 22 for our webinar: "Decoding Data Products: A Blueprint for Business Success in the Age of AI" where Sharad, together with Qlik's Dan Potter and IDC's Chandana Gopal, will explore how data products are transforming industries in today's AI-driven landscape.
We look forward to seeing you in Orlando!