The LLM App Analogy
Today, I’m going to present my favorite analogy on how LLM based applications really work. Generative AI can produce some amazing results and it is a constant source of confusion about how they really work.
Where are the answers coming from? What are the opportunities for improvement? Doesn’t the AI just “learn”?
I’m going to present 3 different scenarios posed as a real-world human interaction. These scenarios are ordered in terms of complexity and capability. In the future, I plan to create a deeper technical dive into each of these scenarios but today’s article will focus on the higher-level concepts and attempt to avoid unnecessary jargon.
In each scenario, the role of the AI is depicted as a well educated man and the app itself is a woman who is trying to get some information from him. They sit at a desk which represents the prompt window and what the AI will have access to answer the questions. The scenarios will differ around what is on the desk, and what types of results to expect.
Level 1: Basic question / answer
Scenario: Nothing is on the desk
In this case, the woman can ask the man lots of questions and he will do his best to answer. He is a well educated person after all, and he probably knows lots of answers or could give opinions on various…
