How Training Actually Works

Welcome back to the NEXUS AI Literacy Series. We've talked about what a language model is, how it turns words into coordinates, and the Transformer engine that lets it weigh how words relate. But there's a giant question we keep gesturing at and haven't answered: how does the thing actually learn? When you hear that a model was "trained," what literally happened? Where do the millions of dollars go? Why does it take months and warehouses full of chips? That's today. By the end, you'll understand training well enough to explain it to anyone — and you'll understand why it's the single most expensive and important step in building an AI.

Let me start with the picture I planted back in Episode 1. A model is a giant network of tiny adjustable dials — we call them parameters — and a big model has hundreds of billions of them. When a model is brand new, before any training, every one of those dials is set to a random position. And a model with random dials is completely useless. You give it the start of a sentence, ask it to predict the next word, and it produces pure gibberish — random letters, nonsense. That's our starting point: a blank network, billions of knobs all set to random, knowing absolutely nothing.

So how do we get from that to something that can write essays and code? Through a process that is, at its heart, almost absurdly simple, just repeated an unimaginable number of times. Let me walk you through one single step of training, because once you understand one step, you understand all of it.

Step one: we take a real piece of text from the training data — say, a sentence from a book. We show the model the sentence up to a certain point and hide the next word. "The cat sat on the ___." Step two: we ask the model to predict that hidden word. With its random dials, early on, it might guess "telephone." Step three — and this is the key — we reveal the real answer. "Mat." Now we can measure exactly how wrong the model was. The gap between what it guessed and the correct answer, that's what we call the error, or the loss. And step four: we go back into the network and nudge the dials — just a tiny bit — in whatever direction would have made the guess a little closer to "mat." Not all the way. Just a hair. Then we do it again with the next piece of text. And the next. And the next.

That's it. That's the whole engine of learning: guess, check against reality, nudge the dials to be a little less wrong, repeat. The technical name for the clever part — figuring out which dials to nudge and in which direction, out of hundreds of billions — is "backpropagation," and the overall steering method is "gradient descent." You don't need the math. You need the intuition, and here's the perfect analogy for it.

Imagine you're standing on a foggy mountainside, blindfolded, and you want to get to the lowest point in the valley. You can't see anything, but you can feel the slope under your feet. So you take a small step in the steepest downhill direction. Then you feel around again, and take another small step downhill. Step by step, always heading down the slope, you eventually reach the bottom. That's gradient descent. The "height" on the mountain is how wrong the model is. The bottom of the valley is being as right as possible. And each step downhill is one tiny adjustment to those billions of dials, nudging them toward less error. Training is just walking downhill in the fog, millions and millions of tiny steps, until the model is about as right as it can get.

Now, here's where the scale becomes staggering, and where the money goes. One step — one guess-check-nudge — barely moves the needle. To get good, the model has to do this not thousands or millions of times, but trillions of times, across a huge fraction of everything humanity has written. It reads, predicts, and adjusts its way through libraries' worth of text, over and over. And remember, every single nudge has to ripple through hundreds of billions of dials. That is an astronomical amount of arithmetic. It's why training a frontier model requires thousands of specialized chips running full tilt for weeks or months, burning enormous amounts of electricity, at a cost that runs into the tens or even hundreds of millions of dollars. The actual idea is simple. The scale is what's brutal.

And something quietly miraculous happens along the way. In the beginning, to get "the cat sat on the ___" right, the model just learns simple word associations. But as it grinds through more and more text, simple associations stop being enough. To correctly predict the next word in a paragraph about, say, a legal dispute, it has to learn how legal arguments work. To finish a line of computer code, it has to learn the rules of that programming language. To complete a sentence of reasoning, it has to learn logic. Nobody adds these skills in as separate lessons. They're forced into existence because understanding is the only way to keep getting the next word right. Knowledge, grammar, reasoning, even a rough sense of how the physical world behaves — all of it precipitates out of that one relentless drill, like crystals forming in a solution. We call these emergent abilities, and they're one of the most surprising things about this whole field: capabilities the builders didn't explicitly design, that simply appear once the model and the training data get large enough.

Now I want to connect this back to a distinction from Episode 1, because this is where it really clicks. Everything I just described — the months, the millions, the trillions of nudges — that is training, and it happens once, up front. When it's finished, you freeze the dials. They stop moving. That frozen network is the finished model. From then on, every time you actually use it — every question you ask ChatGPT or Claude — the dials don't change at all. It's just running your input through the network it already became. That's inference, and by comparison it's cheap and fast. Training is the years of education; inference is showing up and doing the job. This is also why a model has a "knowledge cutoff" — it only knows what was in the text it trained on, up to the day training stopped. It didn't read this morning's news, because its learning ended when we froze the dials.

One more honest point, because it matters for using these tools wisely. The model becomes a reflection of what it was trained on. If the training text carried human biases — and human writing absolutely does — the model can absorb and even amplify them, because it's faithfully learning the patterns it was shown. The training data isn't some pure distillation of truth; it's us, with all our brilliance and all our flaws. That's not a glitch you can fully scrub out. It's a direct consequence of how learning works, and it's why the quality and the curation of training data is one of the most important, and most debated, parts of building these systems.

So let's bring it home. A model starts as a blank network with billions of dials set at random, producing nonsense. Training is one simple loop repeated at incomprehensible scale: show it real text, let it predict the next word, measure how wrong it was, and nudge the dials a hair toward right — like walking downhill in the fog, one tiny step at a time, toward the lowest point of error. Do that trillions of times and not just word-matching but genuine reasoning, knowledge, and skill emerge, because understanding is the only way to keep winning the prediction game. That whole process is enormously expensive and happens once; afterward the dials freeze, and using the model — inference — is the cheap, fast part. And whatever was in the training data, good and bad, is what the model becomes.

So next time you hear "we trained a new model," you'll know what really happened: somebody walked a network of billions of dials down a foggy mountain, trillions of steps, until it stopped being wrong.

In the next episode, we'll tackle those dials head-on — what people actually mean when they brag about "billions of parameters," whether bigger really is better, and how to make sense of all the model sizes you keep hearing about. See you there.