And how the brain you already own does the same job with 20 watts. One long page: a brain you can take apart, a real language model that runs and trains right here, and the path to building our own and giving it away.
A large language model is a very big table of numbers that has been adjusted, billions of times, until it is good at one game: guess the next piece of text. Everything else you see it do (answer, write code, plan) falls out of playing that game extremely well.
Start with the original. About 1.3 kilograms, about 86 billion neurons [1], running on roughly the power of a dim bulb [3]. It is not one machine. It is a set of regions with different jobs, wired together, all running at once.
Below is a teaching model. Hover any region to light it up and see what it does, what the closest part of an LLM is, and where that comparison stops being true. Left alone, it runs a tour: a signal enters through the eye, crosses the cortex, then the lid comes off to show the deep parts.
Pick a region
What it does
Closest thing in an LLM
Where the comparison breaks
You see the word "apple" on a page and say it out loud. That takes about half a second and crosses most of the brain. This is the classic textbook route, simplified:
Notice the shape: input, a stack of processing stages, output, with one part holding the goal the whole time. Keep that shape in mind. A language model has it too.
| # | Part | What it does | Closest thing in an LLM |
|---|---|---|---|
| 01 | Prefrontal cortex | Keeps the goal in mind, plans the steps, and stops the impulse that would break the plan. This is working memory and self-control. | The context window and the system prompt. Whatever sits there steers every word that follows. |
| 02 | Motor and speech cortex | The back of the frontal lobe. Broca's area lines up the words; the motor strip sends the command to the muscles. | The output head: it scores every possible next token, and one is picked and written. |
| 03 | Parietal lobe | Touch, body position and space. It builds the map of where things are and helps point attention at one of them. | Attention: for this step, which earlier pieces of the input matter most? |
| 04 | Occipital lobe | The visual cortex. It turns signals from the eye into edges, then shapes, then objects, in a stack of layers. | The tokenizer and the embedding layer: raw input turned into numbers the rest of the network can use. |
| 05 | Temporal lobe | Hearing, recognising faces and objects, and word meaning. Wernicke's area, at the back of it, is where speech starts to make sense. | The middle layers, where meaning lives as directions in a space of numbers. Researchers can find single features there, such as one for the Golden Gate Bridge. |
| 06 | Cerebellum | Smooths and times movement, and learns from the error between what you meant and what happened. It holds most of the brain's neurons. | The feed-forward (MLP) blocks, and supervised learning: predict, compare with the answer, adjust. |
| 07 | Brainstem | Breathing, heartbeat, sleep and waking. Every signal between the brain and the body passes through it. | The runtime: the servers, power and cables that keep the model running and carry text in and out. |
| 08 | Thalamus | The hub in the middle. Almost every sense passes through it on the way to the cortex, and it decides what gets through. | The residual stream: the shared channel every layer reads from and writes to. |
| 09 | Hippocampus | Binds a new experience into a memory within seconds, then replays it during sleep so the cortex can learn it slowly. | Retrieval memory: notes and documents stored outside the model and looked up when needed. Training is the slow cortex. |
| 10 | Amygdala | Tags an event as a threat or a reward, fast, so that you react and remember it. | The closest thing is a classifier that flags what needs care or priority. |
| 11 | Basal ganglia | Chooses one action out of many and turns repeated choices into habits. Dopamine here signals better or worse than expected. | Reinforcement learning: try an output, get a score, shift the weights toward what scored well. |
| 12 | Hypothalamus | Hunger, thirst, body temperature, hormones and the body clock. It keeps the body inside safe limits. | Nothing. A model has no body, no needs and no drives of its own. |
Zoom all the way in. Every region above is made of the same basic unit: the neuron. It does one thing. It adds up what it hears and decides whether to fire.
A neuron you can drive
This is the famous shape. Every spike from a given neuron looks the same, so the message is not in the size of the spike. It is in how often and when the neuron fires.
One spike · voltage inside the cell, in millivolts
In 1943 Warren McCulloch and Walter Pitts wrote the neuron down as arithmetic [8]. Fifteen years later Frank Rosenblatt built one that could learn [9]. The artificial neuron keeps the core idea and throws away almost everything else:
| Biological neuron | Artificial neuron | |
|---|---|---|
| Inputs | Chemical nudges on thousands of dendrite branches | A list of numbers |
| Connection strength | Synapse strength, changed by use | A weight: one number per input |
| Adding up | Voltage builds in the cell body, and leaks | Multiply each input by its weight, add |
| Decision | Fire fully at about -55 mV, or not at all | A simple curve, such as "keep it if positive, else zero" |
| Output | A train of identical spikes, timed | One number |
| Speed | A spike takes about 2 ms; signals travel metres per second | Billions of multiplications per second on a GPU |
| Energy | The whole brain: about 20 W | One H100 GPU: up to 700 W [27] |
The slider demo above is exactly an artificial neuron wearing a biological costume: three inputs, three weights, a sum, a threshold.
One neuron can only draw a single line between yes and no. Stack them in layers, feed each layer into the next, and the network can represent almost any pattern. The knowledge is not stored in any one neuron. It is spread across the weights of all the connections.
A small network · every line is one weight
Each synapse changes based on what its own two neurons just did. Nothing tells it the big picture. Signals such as dopamine then say, broadly, "that went better than expected, keep those changes" [33]. It works while you live, one experience at a time.
Run an example through. Compare the answer with the right one. Then work backwards through every layer and compute, for every single weight, how much it contributed to the error. Nudge each one the right way. This is backpropagation [10]. No one has found a mechanism in the brain that does exactly this, and whether the brain approximates it is an open question.
A network only does arithmetic, so text has to become numbers first. That happens in two steps.
A tokenizer cuts text into pieces and gives each piece an ID. Real models use pieces smaller than words, found by a method called byte pair encoding [24]: start from single characters, then keep merging the pair that appears together most often. Common words end up as one token; rare words are spelled out of parts. GPT-2 ended up with a vocabulary of 50,257 pieces [16]. For English, a common rule of thumb is about four characters per token.
The model on this page keeps it simple: one token per word, 31 words in total. Type using only its words and watch them turn into IDs. Anything it has never seen is dropped.
The tokenizer of this page's model
Its whole vocabulary:
An ID is just a label. So each token gets its own list of numbers, called an embedding: its coordinates in a space of meaning. Those coordinates start random and are learned during training, like every other weight. Our model uses 16 numbers per word. GPT-3 used 12,288 [17].
Nobody tells the model which words are alike. Below are the real coordinates our model learned, squeezed from 16 dimensions down to 2 so they fit on a screen.
What the model learned about its words · 16 dimensions flattened to 2
Since 2017 nearly every language model has used the same design, the transformer [15]. It is a stack of identical layers. Each layer does two things to every token.
After the last layer, the model scores every token in its vocabulary as a candidate for the next one, turns the scores into probabilities, and picks. Then it adds that token to the text and does the whole thing again.
Here is that loop as a machine. It runs our real model: the lifters are its true attention weights and the bars are its true probabilities.
Attention is the idea that made all of this work. Two prompts that end in the same two words, "on the", need different endings. The only way to get it right is to look back at the animal. Here is where our trained model actually looks:
Where the last word looks before choosing the next one
Nobody programmed that. The model was only ever told "predict the next word". Looking back at the subject is the strategy it discovered because it lowers the error.
The live model · 2,975 parameters
Bar under each word: how much the model is looking at it to choose the next one.
Click any candidate to choose it yourself. Slide the temperature down and the model plays it safe; slide it up and it takes chances.
The model never outputs a word. It outputs a probability for every word, and something has to choose. Temperature reshapes those probabilities before the choice. These are our model's real numbers for what follows the single word "the":
What comes after "the"? · the same scores at three temperatures
Same design, more of everything. Note the scale is logarithmic: each step to the right is ten times more.
Learned numbers, from this page to a brain · log scale
| Model | Year | Parameters | Layers | Numbers per token | Context |
|---|---|---|---|---|---|
| This page | 2026 | 2,975 | 1 | 16 | 8 tokens |
| Transformer, base [15] | 2017 | 65 million | 6 + 6 | 512 | translation |
| GPT-2 small [16] | 2019 | 124 million | 12 | 768 | 1,024 tokens |
| GPT-3 [17] | 2020 | 175 billion | 96 | 12,288 | 2,048 tokens |
| Llama 3.1 [19] | 2024 | 405 billion | open weights, trained on over 15 trillion tokens | ||
Before training, every weight is random and the model outputs noise. Training is one short loop, repeated until the numbers are good.
Press the button. Your browser will create a fresh model with random weights and train it on 16 sentences, right now. The chart is the loss. The three test prompts under it show what the model would say at that moment.
Train a language model in this tab
Random weights. Nothing learned yet.
Three things to notice.
This is the real loss curve from the run that produced the weights you played with above: 1,500 steps, 2.3 seconds.
Loss during the build · lower is better
No library. This is the function that every number on this page comes from.
function forward(w, ids) {
const n = ids.length;
const h = ids.map((id, i) => w.E[id].map((value, j) => value + w.Pos[i][j]));
const last = h[n - 1];
const q = mulVec(w.Wq, last);
const keys = h.map((hi) => mulVec(w.Wk, hi));
const values = h.map((hi) => mulVec(w.Wv, hi));
const scores = keys.map((key) => key.reduce((sum, kj, j) => sum + kj * q[j], 0) / Math.sqrt(D));
const attn = softmax(scores);
const c = Array(D).fill(0);
for (let i = 0; i < n; i++) for (let j = 0; j < D; j++) c[j] += attn[i] * values[i][j];
const z = last.map((value, j) => value + c[j]);
const pre = mulVec(w.W1, z).map((value, i) => value + w.b1[i]);
const m = pre.map((value) => Math.max(0, value));
const out = mulVec(w.W2, m);
const z2 = z.map((value, j) => value + out[j]);
const logits = mulVec(w.U, z2).map((value, i) => value + w.b[i]);
return { h, last, q, keys, values, attn, c, z, pre, m, z2, logits };
}
Scale that loop up by a factor of a hundred million and it stops being a script. It becomes a factory with several teams and a very large electricity bill.
Predict the next token over trillions of tokens. This is where grammar, facts, reasoning patterns and style come from. The result can continue any text but has no idea it is supposed to be a helpful assistant.
Train a little longer on examples of the behaviour you want: question, then a good answer. The model keeps what it knows and learns the format, the tone and when to stop.
People compare answers, and the model is rewarded for the kind they prefer [25]. Some labs also train against a written set of principles so the model can critique and improve its own answers [26]. This is the nearest thing to the dopamine signal in section 3.
| Build | Parameters | Hardware | Time | What you get |
|---|---|---|---|---|
| This page's model | 2,975 | One laptop core | 2.3 s | Understanding |
| nanoGPT, GPT-2 size [20] | 124 million | 8 x A100 | about 4 days | Writes fluent English |
| nanochat [21] | 561 million | 8 x H100, rented | 3 h 51 min, about $92 | A small chatbot, end to end |
| Llama 3.1 405B [19] | 405 billion | over 16,000 H100 | not published | Frontier class, open weights |
One number to feel the gap. An H100 rents for roughly $2 to $5 an hour [28, 29]. A cluster of 16,000 would be about $32,000 to $80,000 for every hour it runs. That is simple multiplication, not a published price, but it shows why only a handful of organisations pretrain at the frontier, and why everyone else builds on top of open weights.
The two are alike enough that each field keeps borrowing from the other, and different enough that calling an LLM "a brain" is wrong.
| Shared idea | Evidence |
|---|---|
| Knowledge lives in connection strengths | True by design in a network, and the standard account of memory in the brain. |
| Prediction is the engine | Language models trained only to predict the next word turn out to predict human brain activity during reading remarkably well. In one study the best models accounted for nearly all of the explainable variance in the recordings [32]. |
| Layers build up from simple to abstract | Vision in the brain and in deep networks both go edges, shapes, objects [12]. |
| Meaning is a direction in a space | Researchers pulled millions of readable features out of one production model, including one for the Golden Gate Bridge [30]. |
| Memory systems look related | A transformer with a certain kind of position code behaves mathematically like models of the hippocampus [31]. |
Power draw · log scale
| Your brain | A large language model | |
|---|---|---|
| Learns | All the time, from single events | Only during training. After that the weights are frozen |
| Memory | Hippocampus writes new memories in seconds [6]; sleep moves them into the cortex [34] | The context window during a chat. Nothing is kept unless someone builds a memory around it |
| Signal | Timed spikes, chemicals, slow waves | Plain numbers, one layer after another |
| Wiring | Loops everywhere; signals run both ways | Strictly one direction per token |
| Data needed | A child learns to speak from the talk around it, a tiny fraction of what a model reads | Trillions of tokens [19] |
| Energy | About 20 W | Megawatts to train |
| Body and needs | Hunger, pain, hormones, a life to protect | None. Its goals come from the person using it |
| Experience | You know there is something it is like to be you | Unknown. Science has no test for this yet, and honest people disagree |
You do not start by pretraining a giant. You start with the cortex someone else already paid for, and you build the missing parts of the brain around it. Then, step by step, you earn the right to train your own.
Take the regions from Fig 1 and ask: which ones does a bare model lack? Each gap is a part you can build today with ordinary software.
| Brain part | What to build | How |
|---|---|---|
| Cortex | A strong model, rented or open | Start here · Rent a frontier model by the token, or run an open one on your own machine. Keep two, so you can switch. |
| Prefrontal cortex | A standing brief in every chat: goals, plan, rules | Start here · One short text file, injected at the top of every conversation: who you are, what is on the plan, how you like work done. |
| Hippocampus | Capture, one fact per note, fast recall | Start here · A folder of small dated notes plus a search over them and over past chats. Pull the few that match before the model answers. |
| Cerebellum | Practised routines the model can run without thinking | Start here · Write each repeated job down once as a step-by-step playbook the model loads when the job comes up. Two of them built this page. |
| Basal ganglia | Habits: your corrections turned into defaults | Next · Every time you correct it, save the correction as a rule with the reason. Load the rules with the brief. |
| Thalamus | A router that sends each job to the right worker | Next · A small first step that reads the request and hands it to the right model, playbook or helper agent. |
| Amygdala | A flag for what needs a human now | Next · A rule that marks money, sends and anything hard to undo as "needs a person", and stops there until someone says yes. |
| Sleep | A nightly pass that replays the day into lasting memory | Next · A scheduled job that reads the day's chats, saves new facts, merges duplicates and fixes what went stale. |
| Our own weights | A small open model tuned on our own way of working | Later · Levels 1 to 4 below: train a small one to learn, then tune an open model on clean examples. |
You are here. The model on this page is complete: tokenizer, embeddings, attention, MLP, training loop, all in plain JavaScript inside this one page. Read its source, save a copy, change a sentence in the corpus, retrain, and see what changes.
view-source:https://anez.digital/llm # search for makeTrainer: the model and its training loop
Rent one 8 x H100 machine for an afternoon and run nanochat from start to finish: about 561 million parameters, three hours and fifty-one minutes [21]. You end with a chat window talking to weights you trained. It will be charming and often wrong, and you will understand every stage of section 7.
git clone https://github.com/karpathy/nanochat cd nanochat # then follow the speedrun in its README
Take a good open model and train LoRA adapters [23] on a few hundred to a few thousand clean examples: real tasks, the answer we would sign. Keep a held-out set so you can prove it got better, not just different.
Build the "next" rows in the table above, above all a nightly digest that turns the day into facts and a clear flag for what needs a human. This gives more than any amount of extra parameters, because it is the part a bare model does not have at all.
A small model with a clear purpose, trained only on text we have the right to use, published with its weights, its code, its test results and its known faults. See the next section.
Recommended next step
Level 1 this month: one afternoon, about $100, and the whole factory stops being a mystery.
| Neuron | A cell that adds up its inputs and fires when they pass a threshold. In a model: one small sum and a simple curve. |
| Synapse · weight | The strength of one connection. Learning means changing these. |
| Parameter | Any number the model learned. Weights are parameters. "405 billion parameters" means 405 billion learned numbers. |
| Token | A piece of text, often part of a word, with an ID number. |
| Embedding | The list of numbers that stands for a token: its position in a space of meaning. |
| Context window | How many tokens the model can see at once. Its working memory. |
| Attention | The step where each token gathers information from the other tokens, weighted by relevance. |
| MLP | The small feed-forward network inside each layer that processes what attention gathered. |
| Layer · block | One round of attention plus MLP. Models stack dozens. |
| Residual stream | The running list of numbers for each token that every layer reads from and adds to. |
| Logits | The raw scores for every possible next token, before they become probabilities. |
| Softmax | The formula that turns scores into probabilities that add up to 100%. |
| Temperature | A dial on the softmax. Low is safe and repetitive, high is varied and risky. |
| Loss | One number for how wrong the model was. Training pushes it down. |
| Backpropagation | The method that works out each weight's share of the blame for the loss. |
| Pretraining | The long first phase: predict the next token over a huge amount of text. |
| Fine-tuning | A short second phase on examples of the behaviour you want. |
| LoRA | A cheap way to fine-tune: train a small add-on and leave the big model untouched. |
| RAG · retrieval | Looking things up in an outside memory and putting them in the context before answering. |
| Inference | Using a finished model. Weights frozen, one token at a time. |
Each one was opened and the number checked on 13 Nov 2026, except those marked classic reference, which are cited from the literature by their DOI.