The Rise of ChatGPT
From the very first moments that OpenAI’s ChatGPT burst on to the scene, commentators have gushed at it’s magical brilliance. The ability to ask questions and receive a comprehensive response, that can be further refined in an ongoing dialog, appears mercurial. Everyone seems both captivated and frightened, but we’re all wondering how we can use it in our business. However, I think it’s a powerful evolution of the humble chatbot that removes many of the adoption barriers I formerly mentioned in Part I.
1. The interaction is effortlessly natural. You just type or dictate your question.
2. There are no predefined commands to learn. It’s just remarkably intuitive.
3. The hesitancy of the unknown is eliminated because users instinctively expect the conversation to iterate until an answer is found. We don’t feel inhibited, but rather the opposite. We want to use ChatGPT.
4. ChatGPT sparks the imagination of how we could use it to our benefit. Enterprise Buddy in comparison felt quite utilitarian.
5. Large Language Models (LLMs) are rapidly evolving to incorporate rich formatting and media, and we are no longer restricted to text answers.
6. Technological advancement in data storage, processing and programming for AI has progressed so much in 20 years that the market is ripe for widespread adoption beyond the OpenAI offerings.
In fact, ex-Google Matt Web posted this quote on his blog:
“It feels pretty likely that
prompting or chatting with AI agents is going to be a major way that we
interact with computers into the future, and whereas there’s not a huge
spread in the ability between people who are not super good at tapping on icons
on their smartphones and people who are, when it comes to working with AI it
seems like we’ll have a high dynamic range. Prompting opens the door for
non-technical virtuosos in a way that we haven’t seen with modern computers,
outside of maybe Excel.”
I whole-heartedly agree!
The Arms Race is On
The AI arms race is definitely in full swing, and we know
that if we can’t work out how to leverage this new technology for our benefit, then
our competitors will. Consequently, the obvious questions are “How?” and “Why?”
I’ve listed three below:
1. Off the Shelf LLM
The low hanging fruit is to subscribe to one of the many
public LLM services and deploy Generative AI chatbots across the organization
as productivity helper to the hundreds of content creators across an
organization. The risk is low, and the rewards are potentially great. Content
creators will be generating emails, web pages, blogs, code aplenty.
2. Extend LLM With Own Data
The natural next step is to extend the concept to include
an organizations’ OWN data. An organization could either host the
infrastructure for LLM’s themselves or augment an existing service as the high-level
diagram below highlights:
But that’s just half of the solution. A chatbot mechanism
is then needed to access the LLM and return responses based on the newly trained
data. The high-level diagram below demonstrates the overall concept:
Enhancing an existing model requires more consideration,
because there’s not only the question of whether a company should host its own
Generative AI infrastructure, but also surfaces issues of data quality and data
security. For example, should all employees have access to a GenAI chatbot if the
organization’s LLM is trained on corporate code-base, documentation, usage
telemetry, HR information, sales data, etc.?
A singular model deployed in this manner could generate answers containing sensitive data to folks not entitled to view it. Consequently, someone in marketing could ask questions about the company’s proprietary source code. Or the same person could ask questions about salaries from HR data. Also, chatbots like Slack and Teams allow group chats, mixing users with different privileges and access. Again, this complicates delivery of a singular Generative AI model. My original chatbot patent had an access and security layer built into the chatbot flow. Therefore, I’d hazard a guess that many corporations and vendors will enforce some sort of question-response access policy as the technology develops and matures.
One other challenge is update methodology. Should the knowledge warehouse be append only, or should old information be deleted, especially as selective pruning of vector databases can be tricky. For example, how do you treat outdated documentation? Do you delete it from the model or just append the new documentation and label it as current? On the one hand having old and new documentation allows for comparative questions to be answered, however answers to general questions based on outdated documentation could be confusing. Perhaps a more specific prompt will lessen this issue. I’ll leave the “prompt engineering debate” to another posting.
3. Embedding Generative AI as Co-pilots
The final place where organizations can quickly benefit from Generative AI is to embed it into their own products as “helpers” or “co-pilots.” Once again, this type of technology is not an original idea. Integratged Development Environments (IDEs) have provided code completion, error detection and refactoring for years, but recently GitHub Copilot took these concepts further by utilizing a large-scale machine learning model to generate swathes of code as a productivity enhancer. Now you can do something similar.
Creating a co-pilot is similar to use case #2 except you train the model on your own code base or business processes and the user interface is a more targeted kind of chatbot. For instance, a social media posting application could provide a ‘co-pilot’ to not only generate interesting posts, but also automate postings at the most optimal time.
Are Chatbots Saved?
I began this post asking the question whether OpenAI and
ChatGPT will save the chatbot industry and I believe this is a resounding YES!
You could argue that chatbots didn’t need saving and were doing just fine. But
the counterpoint is that we’ve not seen the widespread adoption as comparable
innovations in the same period. That was until now. ChatGPT has shaken the IT
industry to its very foundation and sparked innovation on a scale that we’ve
not seen since the invention of the microchip. Chatting with AI agents is going
to be a dominate method of interacting with software in the very near future,
especially as every vendor races to blend and extend Gen AI into their
solutions over the coming six to twelve months.