From Buzzwords to Mental Models: How LLMs Actually Work

1. What Problem LLMs Solve

Before we talk about tokens, embeddings, attention, or even training, it helps to name the job first.

A large language model is not trying to “understand” language the way a person does. Its core job is simpler and stranger: given a sequence of text, predict what is most likely to come next. That sounds narrow, but it turns out to be powerful enough to produce writing, summarize ideas, translate between languages, answer questions, and even simulate reasoning patterns.

Think of it like the fog horn in San Francisco Bay: it does not “know” the whole coastline, but it is tuned to respond to the conditions around it in a way that helps navigation. An LLM is similar. It is not storing a giant encyclopedia in the human sense. It is learning a compact statistical map of how language tends to continue.

That distinction matters. If you think an LLM is a database, you will expect the wrong behavior. If you think it is a chatbot with opinions, you will miss the mechanism. The better mental model is this:

An LLM is a machine for turning context into likely continuations.

Once that clicks, a lot of confusion starts to dissolve.

• Why does it sometimes sound brilliant and sometimes confidently wrong?
• Why do prompts matter so much?
• Why can one model write code, poetry, and summaries?
• Why is training so expensive, and inference comparatively cheap?

All of those questions come from the same root: the model is learning patterns in language, then using those patterns to continue whatever context you give it.

By the end of this course, you should be able to explain this whole system in plain language: what problem an LLM is actually solving, why next-token prediction is enough to create powerful behavior, and how the rest of the pieces fit around that core idea. You will not just know the terms — you will have a mental model sturdy enough to teach someone else.

2. The Core Idea: Prediction

Remember earlier when we said an LLM is a machine for turning context into likely continuations? That sentence is the center of the whole system. Everything else — embeddings, attention, training, inference — exists to make that prediction better.

At first glance, “predict the next token” sounds almost too simple. But simplicity here is misleading. The power is not in the task being complicated; the power is in the scale and quality of pattern learning.

An LLM is like a very advanced autocomplete, except instead of only finishing a word or two, it is learning the deeper shape of how language tends to unfold across sentences, paragraphs, and entire documents.

1) Prediction is a compression of experience

A useful way to think about prediction is this:

Prediction is what you get when a system learns the structure behind many examples.

If you have seen enough language, you begin to notice regularities:

• Questions often invite answers.
• Definitions often follow a pattern.
• Code has syntax that narrows what can come next.
• Stories create expectations and then sometimes break them.

The model does not memorize every sentence as a perfect recording. Instead, it learns a compressed representation of patterns that help it guess what comes next under many different conditions.

Analogy: learning the rhythm of a song

Imagine hearing a song enough times that you can feel when the next beat is likely to land, even if you are not consciously counting. You are not reciting the whole song from memory. You are sensing its structure.

That is close to what prediction means inside an LLM. It is learning the rhythm of language.

This is why the model can sound fluent without “thinking” like a person. Fluency comes from having learned a vast amount of structure that constrains what continuations are plausible.

2) The model is always making a local choice

A key misunderstanding is to imagine the model is planning an entire answer in one shot. It usually is not. It makes one small prediction at a time.

That matters because each step is local:

• given the text so far,
• estimate what token is most likely next,
• choose one,
• then use that as new context,
• and repeat.

This is a little like writing a sentence one word at a time while keeping the rest of the sentence in mind. You do not need perfect foresight to keep moving forward. You need a good sense of what fits right now.

Analogy: crossing a river on stones

Think of stepping across a river on a trail of stones. You do not need to know the exact position of every future stone. You only need to choose the next stable step. Then the next one becomes available from there.

That is what generation feels like inside the model. Each prediction creates the next context, and the next context shapes the next prediction.

This is the same idea showing up again: small decisions accumulate into large behavior.

3) “Likely” does not mean “true”

This is one of the most important distinctions in the whole course.

An LLM is optimized to predict what tends to come next in text. That is not the same as checking whether a statement is factually true.

It can produce:

• accurate facts,
• plausible explanations,
• polished nonsense,
• or a mix of all three.

Why? Because the model is rewarded for continuation quality, not direct truth verification. If the surrounding text makes a claim sound natural, the model may continue in a way that sounds coherent even when the claim is wrong.

Analogy: a skilled actor reading a script

A skilled actor can deliver a line with total confidence, even if the line describes something false. The performance may sound convincing because confidence and truth are different things.

An LLM is similar. It is extremely good at producing the shape of a believable continuation. That is why it can be useful and unreliable at the same time.

Remember earlier when we said an LLM is not a database? This is where that matters. A database retrieves; a predictor continues. Those are different jobs.

4) Prediction depends on context, not just the last word

A simple autocomplete system might look only at the previous word or two. An LLM is different because it can use a much larger context window. In plain language, it can pay attention to a broader stretch of text before deciding what comes next.

That means prediction is not just about local word neighbors. It is about the state of the conversation or document.

For example, the same phrase can imply different next words depending on what came before:

• “The bank approved the…” suggests one kind of continuation.
• “We sat by the bank and watched…” suggests another.

The model is not guessing from a vacuum. It is continuously updating its sense of the situation from the text so far.

Analogy: reading the room

Imagine entering a room mid-conversation. The next thing someone says depends on the tone, topic, and mood already established. If you only heard the last sentence, you would miss a lot. But if you listened for longer, you would infer what kind of response fits.

That is what context does. It gives the model enough surrounding information to make a better prediction.

5) Prediction is not randomness, but it is not fixed either

A common surprise: if a model is predicting probabilities, why doesn’t it always say the exact same thing?

Because prediction usually produces a distribution of possibilities, not just one answer.

Think of it this way:

• some continuations are very likely,
• some are possible,
• some are extremely unlikely.

The model assigns relative weights. Then the generation process may choose the most likely option, or sample from the top options, depending on how it is being used.

So the output is shaped by both:

• what the model has learned, and
• how the next choice is selected.

Analogy: weather forecast, not a prophecy

A forecast saying “there is an 80% chance of rain” is not a guarantee. It is a structured estimate. The model’s prediction works more like that than like a fixed script.

This helps explain why two replies to the same prompt can differ. The model is navigating a space of plausible continuations, not reciting a single locked-in sentence.

6) The model learns the shape of language, not just facts about language

Here is another subtle but crucial point: prediction trains the model on the structure of language itself.

That structure includes:

• grammar,
• style,
• discourse patterns,
• logical transitions,
• common knowledge associations,
• and the ways humans tend to explain things.

This is why LLMs can look like they are reasoning. Often, they are reproducing and recombining patterns that resemble reasoning. That does not mean there is nothing happening internally — but it does mean the foundation is pattern prediction, not symbolic rule execution.

Analogy: learning the grammar of social conversation

You do not need a formal rulebook to know that interrupting mid-sentence can feel abrupt, or that a polite response often follows an invitation. Humans absorb these patterns through exposure.

An LLM does something comparable at massive scale. It learns the statistical regularities of language use, and those regularities include many forms of human thought as they appear in text.

That is why prediction can produce surprising breadth. Language contains traces of explanation, planning, argument, instruction, fiction, and reasoning. Learning to predict language means learning the contours of all of those forms.

A quick mental model

Here is the cleanest way to hold this section in your mind:

If you remember only one thing, remember this:

An LLM is not trying to know everything. It is trying to become very good at continuing language in a way that fits the context.

That one idea explains a lot of the behavior that feels mysterious at first.

It also sets up the next big question: if prediction is the core task, how does the model turn raw text into something it can actually work with? That is where embeddings enter — not as magic, but as a way of turning words into a form the system can compute with.

And that is the bridge to the next part.

3. How the Pieces Build Intelligence

Remember earlier when we said an LLM is a machine for turning context into likely continuations? And then we narrowed that down further: it does this one token at a time, using the text so far as a moving frame of reference.

Now the real question becomes: how does raw text become something a model can reason over at all?

That is where the pieces start to connect.

An LLM is not one giant trick. It is a chain of transformations, where each part does a different kind of work:

• Tokens break text into manageable chunks
• Embeddings turn those chunks into vectors, which are usable by the model
• Attention decides which earlier parts matter right now
• Layers gradually refine the representation
• Training teaches the whole system to predict well
• Inference uses the trained system to generate output

If Section 2 was about the job, this section is about the machinery.

From text to numbers: why the model needs embeddings

A model cannot directly manipulate words the way we do. Words are meaningful to us, but to a machine they are just symbols until converted into numbers.

That conversion is not just a technical convenience. It is the first step toward meaning.

Think of embeddings as coordinates in meaning-space

Imagine every token is placed on a map, not by spelling, but by behavioral similarity.

Words that appear in similar contexts end up closer together. That means:

• “cat” and “dog” may live near each other
• “Paris” may sit closer to “France” than to “banana”
• “run” as a verb may be separated from “run” as a noun in useful ways

So an embedding is not a definition. It is more like an address in a very large abstract city of meaning.

That matters because once words become vectors, the model can do geometry with them. And geometry is something neural networks are very good at.

This is the first big shift: from language as symbols to language as structure.

Remember the “likely continuation” idea from Section 2? Embeddings are one reason that continuation becomes more than matching surface patterns. The model can represent similarity, contrast, and context in a form it can calculate with.

Why attention matters: not all context is equally important

If embeddings are the map, attention is the spotlight.

When the model is deciding what comes next, it does not treat every prior token as equally relevant. Some parts of the context matter a lot; others barely matter. Attention is the mechanism that lets the model look back and ask:

“Which earlier pieces should influence this prediction most?”

Analogy: reading with a highlighter

When you read a paragraph, you do not give every sentence equal weight. You mentally highlight the parts that answer the current question.

If you are trying to resolve a pronoun, you scan for what it refers to. If you are following an argument, you notice the premise that supports the conclusion. If you are reading code, you jump back to see where a variable was defined.

That is attention-like behavior.

It is not just recalling the past. It is selective recall based on current need.

This is a major reason LLMs feel context-aware. The model is not blindly averaging the past. It is dynamically deciding what matters for this moment.

How attention and embeddings work together

Embeddings give each token a position in a mathematical space. Attention then compares those positions to decide relationships.

Think of it like this:

• embeddings say what each token is like
• attention says which tokens relate to each other right now

That division of labor is powerful.

A sentence is not just a pile of words. It is a network of relationships:

• who did what to whom
• what refers back to what
• what contrasts with what
• what depends on what

Attention helps the model build those relationships on the fly.

Example: pronouns become easier

Take a sentence like:

“The trophy didn’t fit in the suitcase because it was too small.”

What does “it” refer to? The trophy or the suitcase?

Humans use context and world knowledge to infer the likely referent. The model does something similar by using attention to weigh the relevant tokens. The point is not that it “understands” like a person. The point is that it has a mechanism for connecting the right parts of the text.

This is the same idea showing up again: context is useful only if the system can identify what matters inside it.

Why layers are needed: one pass is not enough

If attention helps the model look around, layers help it think in stages.

A single transformation of text is usually too shallow to capture everything needed for good prediction. So the model stacks many layers, and each one can refine the representation a little more.

You can think of this as a series of increasingly informed drafts.

• Early layers may capture local patterns: nearby words, grammar, short-range connections
• Middle layers may combine pieces into phrases, roles, and relationships
• Later layers may support higher-level patterns like intent, style, and broader coherence

This does not mean the model literally thinks in neat human stages. But it does mean information gets transformed progressively.

Analogy: developing a sketch into a clear drawing

Imagine a pencil sketch of a building.

At first it is rough: outlines, proportions, basic structure. Then you add windows, shadows, depth, and details. By the end, the drawing is still the same building, but now it is far more informative.

Layers do something similar to the internal representation of text.

A useful way to say it is:

Each layer asks the next, “What more can be made visible here?”

That is why deep models are so effective. They do not rely on one massive leap. They build complexity gradually.

Training: how the model learns the map

So far we have talked about what the pieces do. But how do they become useful in the first place?

Training is where the system gets shaped.

Remember Section 2: the model is trying to predict the next token. During training, it sees huge amounts of text and keeps adjusting itself when its guesses are off.

Over time, the model starts to internalize patterns such as:

• which words fit which contexts
• how syntax constrains likely continuations
• how discourse patterns unfold across longer passages
• which representations help prediction become more accurate

Training is not teaching the model facts one by one. It is tuning a large network so that its internal geometry makes prediction easier.

Analogy: learning to catch a ball

A child does not calculate physics equations before catching a ball. With practice, the body learns the relationship between trajectory, speed, and timing.

Training is like that. The model is exposed to many examples and gradually gets better at anticipating what comes next.

It is not memorizing every situation individually. It is learning the underlying regularities that make many situations predictable.

Inference: the trained system in motion

Once training is done, the model enters inference.

Inference is simply the moment when the trained model is used to generate predictions for a real prompt.

This is where everything from earlier comes alive:

1. Your input is tokenized
2. Tokens are converted into embeddings
3. Attention determines what in the context matters
4. Layers transform the representation
5. The model predicts the next token
6. That token becomes part of the new context
7. The loop repeats

So inference is not a separate mystery. It is just the trained system doing its job repeatedly.

A small but important distinction

Training changes the model’s parameters. Inference uses those parameters.

That difference explains why training is expensive and inference is comparatively cheaper.

• Training is the long process of learning
• Inference is the short process of applying what was learned

If you remember the river-stone analogy from Section 2, training is what placed the stones; inference is the act of stepping across them.

Why the whole system feels intelligent

Now the pieces begin to connect.

The model feels intelligent not because any single part is magical, but because the whole pipeline supports a very rich behavior:

• Embeddings let language become computable
• Attention lets the model focus on what matters
• Layers let it refine ideas progressively
• Training tunes the system to predict well across enormous data
• Inference turns that learned structure into live output

This is why an LLM can answer a question, complete code, summarize a document, or imitate a style. It is not switching into different machines. It is applying the same core mechanism to different contexts.

Think of it like this:

The model is not a library of separate skills. It is one adaptable system whose internal representations can support many tasks.

That is the big aha.

A real-world example: why a support chatbot can sound smart but still fail

Suppose a U.S.-based company uses an LLM-powered customer support assistant.

A customer asks about a refund policy. The assistant may respond fluently because it has learned the language patterns of support conversations. It knows the tone, the sequence, the likely structure of the answer.

But if the policy has a special exception buried in a recent internal update, the model may miss it unless that information is present in the context or connected through another system.

Why?

Because fluency comes from learned language structure, while correct retrieval of specific current policy depends on what context the model can actually see.

This example shows the whole system in miniature:

• The model can generate a convincing answer because it knows the shape of support language.
• It can be wrong because prediction is not the same as verified lookup.
• It improves when relevant context is supplied because attention can focus on that context.

So intelligence here is conditional and engineered, not magical.

And that distinction is important.

The hidden symmetry across the system

One of the nicest ways to understand LLMs is to notice that the same pattern repeats at different levels.

Look at that table closely. It is all one recurring theme:

Take something messy, transform it into a useful representation, then use that representation to make the next decision.

That is the system underneath the system.

Remember earlier when we said an LLM is “context into likely continuations”? Now you can see how the machinery makes that possible.

What you should be able to say now

If someone asked you what makes an LLM work, you should be able to answer without hand-waving:

• It converts text into tokens and embeddings
• It uses attention to decide which parts of context matter
• It processes that information through multiple layers
• It was trained to predict the next token across massive text data
• At inference time, it repeats that prediction loop to generate output

That is the full machine in plain language.

And more importantly, you now know why each piece exists.

The model is not just “doing AI.” It is building a mathematical representation of context that makes next-token prediction increasingly effective.

That is the engine.

In the next section, we can finally ask the question that matters most: so what does this explanation help you do differently? Once the system is clear, the practical implications start to look much less mysterious.

4. Why It Matters Now

You have crossed the point where LLMs are just a collection of buzzwords.

You now have the shape of the system in your head: text becomes tokens, tokens become embeddings, attention chooses what matters, layers refine the representation, training teaches the pattern, and inference uses it to continue the sequence. That is not trivia — that is a working mental model.

What changed for you

You can now look at an LLM and see more than a chatbot.

You can see why it can sound fluent without being grounded in truth. You can see why prompting works: because the model is always responding to context, not just a question label. You can see why more context can help, and why the wrong context can mislead it. Most importantly, you can see that intelligence here is not a single mysterious thing — it is an engineered pipeline for making better next-token predictions.

That shift matters because it changes the kind of questions you ask.

Instead of asking, “Why did it say that?” you start asking, “What context was available, what patterns was it trained on, and where did the prediction path go wrong?”

That is the mindset of someone who understands the system rather than just using it.

What you can do with this understanding

This is where the course becomes practical.

• You can explain LLMs without jargon overload.
• You can debug confusing outputs by tracing them back to context, prediction, and selection.
• You can make better use of prompts because you understand that the model is sensitive to framing and local structure.
• You can evaluate claims about AI more calmly, because you know which parts are about architecture, which parts are about training, and which parts are about inference.

That means you are no longer looking at LLMs as magic. You are looking at them as systems with strengths, limits, and design tradeoffs.

The callback that ties it together

Remember the first idea: an LLM is a machine for turning context into likely continuations.

Remember the second: the core task is prediction, not truth-checking.

Remember the third: the machinery — embeddings, attention, layers, training, inference — exists to make that prediction work at scale.

Put those together, and the mystery shrinks. Not because the model becomes simple, but because it becomes legible.

The identity shift

You now think like someone who sees the system behind the output.

That is a meaningful change. It means you can move from reacting to LLM behavior to diagnosing it. You can tell when a model is being helped by context, when it is being limited by context, and when it is simply generating the most plausible continuation rather than the most accurate one.

That is real understanding.

And once you have it, you do not lose it.

You will notice this model of thinking showing up everywhere: in search, in recommendation systems, in autocomplete, in translation, in any system that turns patterns into predictions. This is the same idea showing up again — just in a new costume.

Leave with this

If you remember nothing else, keep this sentence:

An LLM is a prediction engine built to transform context into the next most likely token.

From that one sentence, the rest of the course falls into place.

You are ready now to explain it, question it, and use it with more clarity than before. And that clarity is the real outcome.