The LLM App Analogy

Byron Salty
6 min readNov 20, 2023

Today, I’m going to present my favorite analogy on how LLM based applications really work. Generative AI can produce some amazing results and it is a constant source of confusion about how they really work.

Where are the answers coming from? What are the opportunities for improvement? Doesn’t the AI just “learn”?

I’m going to present 3 different scenarios posed as a real-world human interaction. These scenarios are ordered in terms of complexity and capability. In the future, I plan to create a deeper technical dive into each of these scenarios but today’s article will focus on the higher-level concepts and attempt to avoid unnecessary jargon.

In each scenario, the role of the AI is depicted as a well educated man and the app itself is a woman who is trying to get some information from him. They sit at a desk which represents the prompt window and what the AI will have access to answer the questions. The scenarios will differ around what is on the desk, and what types of results to expect.

Level 1: Basic question / answer

A man and a woman are conversing over an empty desk.
via DALL-E by author

Scenario: Nothing is on the desk

In this case, the woman can ask the man lots of questions and he will do his best to answer. He is a well educated person after all, and he probably knows lots of answers or could give opinions on various questions or examples. You could ask him to write something for you or generally create something for you.

This by itself is pretty impressive and you can get a lot of great results.

Appropriate prompts:
What is the capital of France?
Can you generate an example CSV file with 10 rows with columns for Name, Age?
Please draft an email to XYZ about ABC.
Tell me a joke about a dog and a bear.

Use Cases:
Public world knowledge (up to a point in time).
Creation of data or code in various formats / output types.
Drafting

Limitations:
However there are some serious problems with our educated person. Importantly he has amnesia and therefore cannot make new memories. He’s also stuck in a room with no access to the outside…

--

--

Byron Salty

CTO. Interested in Leadership, Techology, and People. Focusing on AI and creating cool stuff.