The LLM App Analogy
Today, I’m going to present my favorite analogy on how LLM based applications really work. Generative AI can produce some amazing results and it is a constant source of confusion about how they really work.
Where are the answers coming from? What are the opportunities for improvement? Doesn’t the AI just “learn”?
I’m going to present 3 different scenarios posed as a real-world human interaction. These scenarios are ordered in terms of complexity and capability. In the future, I plan to create a deeper technical dive into each of these scenarios but today’s article will focus on the higher-level concepts and attempt to avoid unnecessary jargon.
In each scenario, the role of the AI is depicted as a well educated man and the app itself is a woman who is trying to get some information from him. They sit at a desk which represents the prompt window and what the AI will have access to answer the questions. The scenarios will differ around what is on the desk, and what types of results to expect.
Level 1: Basic question / answer
Scenario: Nothing is on the desk
In this case, the woman can ask the man lots of questions and he will do his best to answer. He is a well educated person after all, and he probably knows lots of answers or could give opinions on various questions or examples. You could ask him to write something for you or generally create something for you.
This by itself is pretty impressive and you can get a lot of great results.
Appropriate prompts:
What is the capital of France?
Can you generate an example CSV file with 10 rows with columns for Name, Age?
Please draft an email to XYZ about ABC.
Tell me a joke about a dog and a bear.
Use Cases:
Public world knowledge (up to a point in time).
Creation of data or code in various formats / output types.
Drafting
Limitations:
However there are some serious problems with our educated person. Importantly he has amnesia and therefore cannot make new memories. He’s also stuck in a room with no access to the outside world.
He can’t tell you your name. You can’t teach him new things. He doesn’t know current events. He can’t even tell you the time or date. He can’t do math very well. He could write code for you but doesn’t have any way to execute it and give you results.
“But wait!” you say. “ChatGPT remembers my name and the conversation.”
ChatGPT is able to do that because it is operating at Level 2 (at least).
Have you ever witnessed an hallucination — the phenomenon where AI makes up some an alternate reality / set of facts? If so then what you are really experiencing is what happens when the LLM doesn’t know something.
When this happens regarding real world facts it is problematic because you may not know if these “facts” are true or false. But when this happens in the context of a long conversation with an LLM, it is extremely jarring because it is making up facts about things you recently talked about.
The LLM simply doesn’t remember the conversation so it made up a logical history to fit what it does know, but this history is wrong and doesn’t make logical sense to the human involved who easily detects the incongruency.
Level 2: Retrieval Augmented Generation (RAG)
Scenario: The man and woman sit at the desk but now there are a bunch of pieces of paper on the desk that the woman has provided. There is a fair amount of desk space but it’s not unlimited.
First the amnesia problem can be nearly solved by simply writing down the conversation on one of the pieces of paper and putting that on the desk.
The man will read over all the papers and then combine that with his base knowledge to answer any question to the best of his ability.
Another piece of paper most likely contains some basic contextual information if known about the user — name, location etc.
But the real power is that the woman can put other papers POSSIBLY related to the question on the desk and they will be used. These papers could come from intranet sites, pages in specific books or manuals, or any other source of data. In this way, she is augmenting his existing knowledge in a way that allows him to answer that specific question.
However after that question is satisfied there will be no memory of the information or response, unless it is also captured or summarized and put on another paper for future use.
Appropriate prompts:
What are our company holidays?
Who won the last week’s election?
Do you remember my name and where I live?
Summarize the project status for my 10am meeting
Use Cases:
Answering questions about private sources of information
Answer questions with no cutoff date
Conversations with recall / history
User specific conversations with settings / context
Create responses in styles or with previous examples as guides (multi-shot)
Limitations:
Desk space is limited so things like amount of history that can be kept are limited and hallucinations happen.
Perhaps more limiting is that everything that the man wants to know has to be placed on the desk in front of it. The woman has to guess what would be useful and put it on the desk to be used.
Level 3: LLM Agents
Scenario: Now the desk not only has pieces of paper on it like before, but it also holds various tools that the man can use to answer questions. Things like a calendar, clock, calculator, books, a phone, a computer to look things up or even make changes.
The superpower of agents is that they can actually DO things. Not only is our fictional man fed information by the woman, he could go get information himself directly. Not only is he going to give her a response that she could act on, he could directly take action himself!
Any LLM can generate code for you, an Agent can actually execute that code as well.
Appropriate prompts:
Create and send a project summary to everyone in my 10am meeting
Can you mark all of my todo items as complete
What time it is in my city?
What is half of the number of people that live in Chicago?
Use Cases:
Interactions with internal systems / APIs
Answers that require math
Search in external systems
Code execution
Limitations:
The man still has amnesia and is unable to truly learn. It is possible to get better results over time by improving parts of the system to put better data in front of the man or increase and improve tools.
Be on the lookout for my follow-up articles that will go into each of these levels in detail as well as implement an example of each of them step by step.
