From Tokens to Truth: A Mental Model of LLMs

1. What Problem an LLM Is Actually Solving

What an LLM is really doing

At the most basic level, an LLM is not trying to be intelligent in the human sense. It is solving a much narrower, but very powerful, problem:

Given the text so far, what text is most likely to come next?

That sounds almost too simple. But this is the key that unlocks everything else.

Think of it like the foggy edge of San Francisco’s bay: when the view is partial, you do not need a full map of the world to continue moving—you need the best next landmark, the most likely continuation from where you already are. An LLM works the same way with language. It does not “know” in the way people mean knowing. It predicts continuations so well that the result can look like understanding, reasoning, writing, and even creativity.

So when you ask a question, the model is not reaching into a database of facts and selecting an answer. It is using patterns learned from huge amounts of text to estimate what sequence of words should follow your prompt. That means its behavior is shaped by three things:

• The prompt you give it
• The patterns it learned during training
• The constraints of next-token prediction

This framing changes how you interpret every surprising behavior. If it sounds confident but is wrong, that is not a random failure—it is a prediction system producing the most plausible continuation. If it seems insightful, that insight comes from the structure of language patterns it has absorbed. If it changes tone or style, it is because those patterns are also part of the prediction problem.

By the end of this course, you should be able to explain that difference clearly: not just what an LLM outputs, but what problem it is optimizing, why that matters, and why so many of its strengths and weaknesses come from the same core mechanism.

Keep that anchor in mind: an LLM is a next-text prediction system that becomes useful because language itself contains a lot of structure about the world.

2. The Core Idea: Prediction, Representations, and Attention

The core idea: prediction becomes structure

Remember earlier when we said an LLM is a next-text prediction system? That is the anchor. But prediction alone does not explain how it becomes so capable. The missing piece is that the model does not treat text as a flat string of words. It turns language into internal representations—compressed patterns that help it keep track of meaning, context, and relationships.

A useful way to think about this is a music app building a playlist from your listening history. It is not storing one giant opinion about music. It is learning patterns like: if you listened to this artist, you may also like this mood, this tempo, this sequence. The system is constantly making a better guess based on structure it has already internalized. An LLM is similar, except the “playlist” is made of words, ideas, and relationships between them.

That means three ideas matter more than almost anything else:

1. Prediction — what comes next?
2. Representations — what does the model internally keep track of?
3. Attention — what parts of the context matter right now?

These are not separate tricks. They are different views of the same machine.

1) Prediction is not guessing randomly

At surface level, prediction sounds like simple autocomplete. But in practice, the model is making a prediction from a very high-dimensional pattern space. It is not asking, “What word feels right?” It is asking, “Given everything so far, what continuation best fits the patterns I have learned?”

That is why the model can sound intelligent. Language is full of hidden regularities:

• topics introduce likely follow-up topics
• questions imply certain kinds of answers
• formal writing has different continuations than casual writing
• a story opening creates expectations about plot and tone

So prediction is powerful because language is structured. The model is learning that structure.

Think of a chess player looking at a position. They are not memorizing one move. They are recognizing patterns: the king is exposed, the center is open, this piece is pinned. Then the likely good move emerges. In the same way, an LLM sees a prompt and its internal state reflects many learned patterns that make some continuations more likely than others.

Callback: This is the same idea showing up again: the model is always working from partial information and choosing the most plausible continuation.

2) Representations are compressed meaning

If prediction is the goal, representations are the model’s working memory.

A representation is not a dictionary definition. It is more like a mental sketch. When a model reads text, it gradually builds internal vectors—numerical summaries of what seems important. These vectors carry information about things like:

• what topic is being discussed
• what role a word is playing
• whether two ideas are related
• what the current tone or task seems to be

A good analogy is packing for a trip. You do not bring your entire house. You compress what matters into a suitcase. The suitcase is smaller than the full reality, but it preserves the useful structure. That is what representations do.

This explains something subtle: an LLM does not “store” meaning like a human memory does. Instead, it creates a temporary internal picture of the text it is processing. That picture can be rich enough to support surprisingly complex behavior.

For example, the word bank can mean a financial institution or the side of a river. The model does not solve this by looking up a fixed definition. It builds a representation from the surrounding words. If the text mentions deposits, interest, or loans, one meaning becomes more likely. If it mentions water, boats, or erosion, the other meaning becomes more likely.

That is the real power of representations: they allow the model to make meaning context-dependent.

Why this matters

If you only think of an LLM as “predicting words,” you miss the hidden layer where those words become structured concepts. The model is not just reacting to the last token. It is continually carrying forward a compressed state of what the conversation seems to be about.

This is why it can stay on topic across many sentences. The representation acts like a temporary map.

3) Attention decides what matters now

Now comes the mechanism that makes the whole system feel coordinated: attention.

Attention is the model’s way of deciding which earlier parts of the context should influence the current prediction most strongly. Not every word matters equally. If you ask, “What did the scientist say about the experiment?” the model needs to focus on the sentence that mentions the scientist and the experiment, not every unrelated detail that came before.

An easy analogy is being in a crowded room and turning toward one voice. You are not deleting the other voices. You are just making one signal more important for the moment. That is attention: selective emphasis.

Another way to picture it is a work meeting with notes spread across the table. When a new question comes up, you do not reread every page equally. You look toward the page that seems most relevant. Attention is the model’s built-in version of that selective looking.

This matters because language is full of dependencies across distance. The subject of a sentence may appear far before the verb. A pronoun may refer back several lines. The answer to a question may depend on a detail introduced much earlier. Attention helps the model connect those pieces.

Remember: prediction tells the model what to do, representations tell it what is going on, and attention tells it what to use right now.

How these three ideas work together

The easiest way to understand the system is to see the loop:

Taken together, they explain why an LLM can feel so fluent.

It does not need perfect human-like understanding to produce useful language. It needs a representation of context rich enough to support the right prediction, and attention that can retrieve the relevant parts of that context at the right time.

That is why the system is both impressive and limited.

• It is impressive because language is full of patterns that can be compressed and reused.
• It is limited because the model’s world is still built from pattern recognition, not grounded lived experience.

This is the same idea showing up again: the model is powerful when the context contains enough structure for its internal representations to work with.

The hidden consequence: behavior emerges from relationships

One of the biggest misunderstandings about LLMs is assuming they contain little labeled facts inside them, like files in a cabinet. That is not the right picture.

A better picture is a city map of connections. The model knows where roads tend to lead, which neighborhoods are related, and which routes become likely depending on where you start. It is the relationships that matter.

That is why LLM behavior can surprise people. A small change in wording can shift attention, change the internal representation, and produce a very different continuation. The model is not pulling an answer from one fixed shelf. It is navigating a web of relationships.

So when you see the model:

• summarize a long passage,
• answer a question in a different style,
• continue a story coherently,
• or translate between forms,

what you are really seeing is the same machinery operating on different patterns.

Prediction gives direction. Representations give context. Attention gives focus.

And together, they make language feel alive.

If Section 1 told you what an LLM is doing, this section explains how that simple objective becomes something much richer.

Remember that. It will matter even more when we get to training and why the model’s behavior is shaped long before you ever type a prompt.

3. How the System Becomes Behavior

How the system is built from data to tokens to behavior

If Section 1 gave us the goal and Section 2 gave us the inner machinery, this is where the machine starts to feel real. The key shift is this:

An LLM does not begin as a useful assistant. It becomes one through a chain that turns raw text into token patterns, token patterns into learned weights, and learned weights into behavior.

That chain is the bridge between “I know what prediction is” and “I can now explain why the model acts this way.”

From language to tokens

Before the model can learn anything, text has to be broken into pieces it can process. Those pieces are called tokens.

A token is not always a word. It might be:

• a whole word
• a subword
• punctuation
• part of a word

So instead of seeing language as a smooth sentence, the model sees it as a sequence of units. That sounds like a technical detail, but it changes the whole picture. The model is not handling “meaning” directly. It is handling symbolic chunks that can stand in for meaning.

A useful analogy is a sheet music system. Music is not stored as a single continuous feeling. It gets broken into notes, durations, and rests. Those symbols are simpler than the experience of music, but they let a system learn structure. Tokens do something similar for language.

This is why tokenization matters:

• it determines what the model can easily notice
• it affects how efficiently it processes text
• it shapes what kinds of patterns are easy or hard to learn

Remember earlier when we said the model is always predicting continuations? Tokens are the units of that prediction. The model is not guessing “the next idea” in some abstract sense. It is predicting the next token, then the next, then the next. Those token-by-token decisions accumulate into something that looks like a sentence, then a paragraph, then a coherent answer.

From tokens to training signal

Now the important question: how does a model learn which token should come next?

It learns from data, but not by being told explicit rules. It learns by exposure to huge amounts of text and by repeatedly being checked against the actual next token in that text.

That means the training process creates a very simple pressure:

• the model makes a guess
• the real continuation is compared against that guess
• the model adjusts itself so future guesses are better

This is where the system starts to look less like a chat assistant and more like a student practicing by correction.

Imagine a student reading millions of sentence fragments and constantly being asked, “What comes next here?” Over time, the student would start to absorb:

• grammar
• style
• common facts
• narrative patterns
• domain-specific phrasing
• regularities in reasoning-like text

That is not because the student memorized a rulebook. It is because repeated correction shapes what they become sensitive to.

What the model is really learning

The model is not learning one fact at a time in the way a database stores records. It is learning statistical structure:

• which words tend to appear together
• which kinds of contexts imply which kinds of continuations
• which patterns signal questions, explanations, warnings, or examples
• how language itself organizes information

This is why a model can seem to “know” many things. It has absorbed the shape of how people talk about those things.

Weights: the hidden memory of training

The result of training is a huge set of numerical parameters, often called weights.

These weights are the model’s learned internal settings. They are not readable like a list of facts, but they encode a massive amount of pattern sensitivity.

A good analogy is a large mixing board in a recording studio. Each dial affects how a sound is shaped. No single dial contains the song, but together the settings determine the final audio. In the same way, no single weight contains a fact, but the whole network of weights determines how the model transforms input into output.

This is why trained models feel so different from untrained ones.

An untrained model has the same general architecture, but its weights are basically random. It has the machinery without the accumulated experience. Training is what turns a generic system into one that can:

• continue text fluently
• follow conversational patterns
• reproduce styles
• answer with useful structure

So when we say the model has “learned,” we do not mean it learned like a human remembers. We mean its weights now carry the influence of many repeated predictive corrections.

Why the model behaves like the data it saw

Here is one of the most important ideas in the whole course:

A model’s behavior is not just a reflection of its architecture. It is a reflection of the data and objectives that shaped its weights.

This is why models can sound confident, polite, cautious, or biased in ways that mirror their training environment.

If the training data contains many examples of clear exposition, the model gets better at clear exposition. If the data contains many arguments, disclaimers, and hedged statements, it learns that style too. If the data contains contradictions, noise, or gaps, the model also absorbs those patterns.

So the model is not simply “thinking.” It is reproducing a learned landscape of textual behavior.

That does not make it useless. It makes it legible.

Once you understand this, you stop treating every output as a mysterious act of intelligence and start seeing it as the outcome of a system trained on the structure of human language.

Why the same model can answer and hallucinate

This is where people often get confused. If the model learned so much from text, why does it sometimes make things up?

The answer comes from connecting Sections 1 and 2.

The model’s job is still prediction. It is not asked to prove truth in the way a scientist would. It is asked to produce the most plausible continuation given the context.

That means two things can both be true:

• the output sounds linguistically excellent
• the output may be factually wrong

Why? Because plausibility is not the same as verification.

If the surrounding text makes a certain answer feel likely, the model may produce it even if it is not grounded in reality. This is not a bug in the narrow sense. It is a consequence of the training objective.

Think of it like a very fast improv actor who has seen a thousand plays. The actor is great at continuing the scene. But unless there is a reality-checking mechanism, the actor may commit to a line that fits the scene beautifully but does not match the actual truth.

That is the same system showing up again: prediction can generate fluency without guaranteeing correctness.

From training behavior to prompt behavior

At this point, a subtle but important distinction appears.

Training shapes the model’s general tendencies. A prompt shapes the model’s local direction.

Training gives the model its default landscape. Prompting chooses where in that landscape the conversation begins.

This is why the same model can behave differently depending on how you ask:

• a direct question may lead to a concise answer
• a detailed prompt may trigger a structured explanation
• a role-based prompt may activate a style pattern
• a contradictory prompt may create uncertainty or drift

The prompt does not rewrite the model. It steers it.

You can think of the trained weights as the terrain and the prompt as the starting point. The path that follows depends on both.

A US-based example

If someone at a U.S. healthcare startup asks an LLM to summarize a benefits policy, the model may produce a polished and helpful answer. But if the policy includes state-specific exceptions, eligibility rules, or legal wording, the model may still miss a critical detail unless that detail is clearly present in the prompt or context. The model is excellent at structured continuation, but it is still constrained by what the text makes salient.

That example matters because it shows the difference between knowing how to speak about a domain and reliably grounding every claim in that domain.

Why token-by-token generation matters to behavior

Now let’s connect everything.

The model does not generate an answer all at once. It generates one token at a time, each new token influenced by:

• the prompt
• the current internal representation
• the attention over the context
• the learned weights from training

That means behavior is incremental, not prewritten.

This is why small changes can have large effects. One token can change the direction of the next token. One sentence can shift the topic. One framing choice can change whether the model sounds cautious, creative, technical, or uncertain.

The system is like a river that keeps deciding where to flow next based on the shape of the terrain immediately around it. It is not choosing from the top of the mountain once and for all. It is constantly re-evaluating the next step.

That is also why long responses can drift. Each step is locally reasonable, but the overall path may wander if the model loses the thread.

The full chain, all at once

Let’s put the pieces together in one compact view:

This is the full bridge from data to behavior.

And once you see it, the mystery fades a little.

An LLM is not magic. It is not a database. It is not a person. It is a prediction engine whose behavior has been shaped by enormous amounts of text and then steered by whatever context you give it.

Remember earlier when we said attention decides what matters now? That idea becomes much more concrete here. Attention is the mechanism that helps the model use the right parts of the prompt while token-by-token generation keeps steering the answer forward.

That is the system.

And now, when you see an answer, you can ask a much better question than “Is it smart?”

You can ask:

• What did the training likely teach it?
• What in the prompt is steering it?
• What token-level continuation is it trying to maintain?
• Where might plausible language be standing in for actual grounding?

That is the shift from surface understanding to systems understanding.

By now, the parts are no longer separate. Data, tokens, training, weights, prompting, and behavior are all pieces of one loop.

And that is exactly the loop we need in the final section, because once you can see how the system is built, the last question becomes the most useful one of all: what should you expect from it in the real world, and how do you think with it clearly?

4. From user to model: why this understanding matters

You now have the kind of understanding that changes how you see an LLM, not just how you describe one. You know it is a next-token prediction system, you know it builds internal representations, you know attention decides what matters in context, and you know training turns all of that into the weights that shape behavior. That is not a list of facts anymore — that is a working mental model.

The shift is bigger than it looks

The biggest change is this: you no longer have to treat the model like a mysterious conversational box. You can now think like someone who understands the forces behind the output.

When the model answers well, you can recognize the pattern: the prompt gave it enough structure, attention found the right context, and the learned weights produced a fluent continuation. When it answers poorly, you can diagnose the failure without guessing wildly: maybe the context was under-specified, maybe the model was following plausibility instead of truth, or maybe the prompt pulled it toward the wrong region of its learned space.

That means you are no longer just a user. You are becoming a reader of model behavior.

Remember the three anchors

• Section 1: the model is solving next-text prediction.
• Section 2: prediction works because representations and attention organize context.
• Section 3: training, tokens, and weights shape the behavior you actually see.

Put those together and the system stops being magical. It becomes legible.

That is why this understanding matters. You can now explain why an LLM can sound confident without being grounded, why it can be flexible across tasks, and why tiny prompt changes can create big output differences. You understand the difference between fluency and truth, between pattern sensitivity and world understanding.

That is the moment you start thinking like someone who can diagnose the system instead of merely using it.

What this unlocks

This new model of the system gives you three practical powers:

• Better interpretation — you can tell when the model is likely continuing a pattern versus giving a reliable answer.
• Better prompting — you can shape context with more intention, because you understand what the model is actually responding to.
• Better judgment — you can decide when the model is useful, when it needs verification, and when it should not be trusted alone.

That is a real capability shift. Not “I can talk to an AI,” but I can reason about how this system behaves.

The final reframe

If you remember one thing, make it this:

An LLM does not understand the way a person understands. It produces useful language because it has learned the structure of language so well that it can approximate many forms of understanding.

That distinction is the key to everything else.

You have now moved from asking, “What is it doing?” to asking, “What forces are shaping this output?” That is a much stronger place to stand.

And once you think that way, you do not just use LLMs differently — you read them differently. That is the real win.