How large language models work
The machine that learned to talk by predicting the next word, a trillion times over.
A large language model is a neural network trained to predict the next token in text. Pretrained on a huge slice of the internet with that one simple objective, a transformer learns grammar, facts and reasoning patterns; it then generates text autoregressively, one sampled token at a time. Instruction tuning and RLHF turn that raw predictor into a helpful assistant — but it still hallucinates, because fluency is not truth.
Tokenize the text
Raw text is first broken into tokens — subword chunks like whole words, word-pieces, or punctuation — and each maps to an integer ID. The model never sees letters or words directly; it only ever consumes and produces sequences of these token IDs. This is the alphabet the whole system is built on.
Pretrain by predicting the next token
During pretraining the model is shown enormous amounts of text and asked, over and over, to predict the next token given everything before it. Because the correct answer is just the token that actually came next, the text is its own answer key — no human labelling required, which is what lets training scale to trillions of tokens. To predict well it is forced to internalise grammar, facts and reasoning patterns.
Inside the transformer
The engine doing the predicting is a transformer, a deep stack of layers whose key mechanism is self-attention. Attention lets every token look at and weigh every other token in the context, so the model can resolve references and track meaning across long passages. Each token is represented as an embedding vector that these layers repeatedly refine.
Turn a predictor into an assistant
A freshly pretrained model is a raw text-completer, not an assistant — ask it a question and it might reply with more questions. Instruction tuning (supervised fine-tuning on instruction/response pairs) teaches it to follow requests, and RLHF — reinforcement learning from human feedback — further nudges it toward answers people rate as helpful, honest and harmless. This alignment step is what separates a base model from a chat model.
Generate autoregressively
At inference the model generates autoregressively: it predicts a probability distribution over the next token, picks one, appends it to the input, and runs again — repeating until it emits a stop token. Every word you see was produced one token at a time, each conditioned on all the text so far. There is no plan drawn up in advance; the sentence unfolds token by token.
Sample the next token
The model outputs probabilities, not a single fixed word, so a sampling strategy decides which token to actually take. Temperature flattens or sharpens the distribution — low is focused and repetitive, high is diverse and creative — while top-p (nucleus) sampling restricts the choice to the smallest set of tokens covering most of the probability mass. These knobs trade determinism for creativity.
Emergent abilities and hallucination
Scaled up far enough, models show emergent abilities — skills like multi-step arithmetic or translation that weren't explicitly trained and appear rather suddenly with size. But the same machinery has a hard limit: the model optimises for plausible text, not truth, so it states confident falsehoods — hallucinations — with no internal signal that it is wrong. Knowing this is why grounding techniques like RAG exist.
What a language model actually is
At its core, a large language model does something almost comically simple: it predicts the next word. More precisely, given a sequence of tokens (chunks of text), it outputs a probability distribution over which token comes next. That is the entire objective. Everything else people find astonishing — writing code, explaining jokes, drafting legal arguments — is a byproduct of getting extraordinarily good at that one prediction task.
The reason this simple objective produces such rich behaviour is subtle but powerful. To predict the next token accurately across billions of wildly different passages, a model cannot just memorise; it has to build internal representations of grammar, facts about the world, writing styles, and even reasoning patterns. If the text reads "The capital of France is", the only way to reliably continue with "Paris" is to have encoded that fact. If it reads a block of Python ending mid-function, continuing correctly requires a model of how code works. Prediction, pursued relentlessly at scale, forces understanding as a side effect.
This is the mental model to hold onto: an LLM is a next-token prediction engine. It is not a database it looks facts up in, and it is not a person who reasons and then speaks. It is a statistical model of language so large and well-trained that competent prediction and genuine capability become hard to tell apart.
Pretraining: one objective, the whole internet
Pretraining is where the model acquires its capabilities, and it works by self-supervised learning. Ordinary supervised learning needs humans to label data, which is slow and expensive. Language modelling sidesteps this entirely: take any piece of real text, hide the next token, and ask the model to guess it — the true answer is simply the token that actually came next. The data labels itself, which is exactly why training can scale to trillions of tokens scraped from the web, books and code.
Mechanically, the model makes a prediction, its guess is compared to the real next token, and the error is used to nudge billions of internal weights via gradient descent. Repeat this trillions of times and the weights slowly settle into a configuration that predicts language remarkably well. The scale involved is staggering — the largest models have hundreds of billions of parameters and cost millions of dollars in compute to train — and one of the field's central findings is the existence of scaling laws: predictable improvements in capability as you add more data, parameters and compute.
The output of pretraining is a base model: a raw, immensely knowledgeable text-completer. It has read a huge slice of humanity's writing and can continue almost any text plausibly. What it cannot yet do is behave like a helpful assistant — that comes later.
Inside the transformer
The architecture that made modern LLMs possible is the transformer, introduced in 2017. Its defining ingredient is self-attention. Older sequence models processed text word by word and struggled to connect ideas far apart; attention lets every token directly look at, and weigh the importance of, every other token in the context at once. When the model reads "the trophy didn't fit in the suitcase because it was too big", attention is what lets it figure out that "it" refers to the trophy.
Before any of this, each token ID is converted into an embedding — a vector of numbers that positions the token in a high-dimensional "meaning space" where related concepts sit close together (the companion concept on how LLMs see the world covers this). The transformer's stacked layers then repeatedly transform these vectors, each layer mixing information between tokens via attention and refining the representation, until the final layer produces the probability distribution over the next token.
Two properties made the transformer win. It is highly parallelisable — unlike its predecessors it can process a whole sequence at once, which is what makes training on internet-scale data feasible on modern GPUs. And attention gives it a long effective memory, so it handles long-range dependencies far better. Those two traits together are why virtually every leading LLM is a transformer.
From predictor to assistant
A base model is powerful but unruly. Ask it "What is the capital of France?" and it might, quite reasonably, continue with a list of similar quiz questions — because in its training data, questions are often followed by more questions. Turning this raw predictor into something that helpfully answers takes a second phase of training, usually in two steps.
First is instruction tuning (supervised fine-tuning): the model is trained on a curated set of instruction-and-response examples, teaching it that when given a request, the desired continuation is a helpful, direct answer. This single step dramatically changes the model's default behaviour from "complete this text" to "respond to this request".
Second is RLHF — reinforcement learning from human feedback. Humans rank multiple model responses from best to worst; those rankings train a separate reward model that scores responses, and the LLM is then optimised to produce answers the reward model rates highly. This is how models are steered toward being helpful, honest and harmless — refusing dangerous requests, hedging appropriately, and adopting a consistent tone. The gap between a base model and the polished assistants people actually use is almost entirely this alignment work.
Generation, one token at a time
When you chat with an LLM, it produces text autoregressively: it predicts a distribution over the next token, selects one, appends it to the running sequence, and feeds the whole thing back in to predict the token after that. The response you read was assembled one token at a time, each choice conditioned on everything generated so far, until the model emits a special stop token.
Crucially, the model doesn't output a word — it outputs probabilities, and a decoding strategy decides which token to actually pick. Always taking the single most likely token (greedy decoding) sounds safe but produces flat, repetitive text. So instead we sample, and two knobs shape how:
Temperaturescales the sharpness of the distribution. Near 0 the model becomes nearly deterministic and focused; higher values flatten the odds, making rarer tokens more likely and output more varied and creative — sometimes to the point of incoherence.Top-p(nucleus) sampling only considers the smallest set of top tokens whose probabilities sum top(say 0.9), sampling from that pool. This adapts the candidate set to the model's confidence, cutting off the long tail of nonsense while preserving genuine variety.
Because generation is sequential, it is also inherently latency-bound: a long answer takes many forward passes, which is why streaming responses token by token feels natural.
Emergent abilities and the limits
Two things about LLM behaviour deserve emphasis, one exciting and one sobering. The exciting one is emergence: certain abilities — multi-step arithmetic, translating languages, following novel instructions — are largely absent in small models and then appear, sometimes abruptly, once a model crosses a size threshold. Nobody explicitly programmed these skills; they materialised from scaling up the same next-token objective, which is a large part of why the field kept building bigger models.
The sobering one is hallucination. Because a model is trained to produce plausible, fluent text — not true text — it will confidently generate falsehoods: invented citations, fake API methods, wrong dates, all delivered in the same authoritative tone as correct answers. There is no internal fact-checker and no built-in signal for "I don't actually know this." Fluency and truth are simply different objectives, and the model was only ever optimised for the former.
Understanding this limit is what motivates much of applied AI. RAG grounds answers in retrieved source documents so the model quotes real text instead of guessing. Tool use lets it call a calculator or search engine rather than fabricating. And human oversight remains essential wherever correctness matters. The takeaway is a balanced one: LLMs are extraordinary pattern machines whose fluency can outrun their reliability, and using them well means designing around both the power and the blind spots of a system that, underneath it all, is just predicting the next token.
The core objective is absurdly simple: predict the next token given the ones so far.
Pretraining is self-supervised — the text is its own answer key, so no human labels are needed.
The transformer's attention lets every token weigh every other, which is why it captures long-range meaning.
Generation is autoregressive: the model samples one token, appends it, and predicts again.
Sampling knobs (temperature, top-p) trade determinism for creativity.
Instruction tuning + RLHF align the raw predictor into a helpful assistant; hallucination remains a core limit.
Quick quiz
What objective is a language model actually trained on?
What does RLHF add on top of a pretrained model?
How the Transformer architecture works
The architecture behind GPT, Claude, Gemini and Llama — one block at a time.
How LLMs see the world (tokens & embeddings)
To a model, your prompt isn't words — it's a list of numbers with a strict budget.
AI vs ML vs deep learning vs generative AI
Four nested circles that get thrown around interchangeably — untangled with one example each.
Fine-tuning vs LoRA vs RAG
Three ways to specialize an LLM — and when each one is the right call.