How neural networks compute
Every layer in a neural network is arithmetic: multiply, add, squash. This page lets you watch that arithmetic happen one number at a time, starting with the layers that let a network see images, and ending with the mechanism inside large language models.
1Convolution
A convolutional layer slides a small grid of weights, the kernel, across an image. At each position it multiplies every weight by the pixel underneath, adds up the nine products, and writes that one number into a new grid called a feature map.
The kernel's numbers decide what it notices. A kernel with negatives on the left and positives on the right gives a large output only where the image gets brighter from left to right, so it finds vertical edges. In a trained network nobody writes these numbers by hand; they're learned.
The same nine weights work on a real photograph. The only difference is scale: a 160×160 image means 25,600 positions instead of 64.
2Pooling
A feature map is as large as the image it came from. Pooling shrinks it. A window slides over the map and keeps one number per window, either the largest (max pooling) or the average. Nothing is learned here; there are no weights.
Shrinking is useful. Later layers have fewer numbers to process, and a feature that shifts by a pixel usually stays in the same window, so the network cares less about exactly where something is.
3Dense networks
After convolution and pooling, a CNN usually ends with a few dense layers, and on their own they make up the plain neural network. Every neuron reads every output of the layer before it:
z = w1a1 + w2a2 + … + b, a = f(z)
The weighted sum z goes through an activation function f, which bends the line. Without that bend, stacking layers would collapse into one big linear function.
4Recurrent networks
Every network so far sees its whole input at once. Text, speech and time series arrive in order, so a recurrent network reads one item at a time and keeps a small vector h, the hidden state, that carries what it has seen so far:
ht = tanh( Wxhxt + Whhht−1 + b )
The same weights are used at every step. Drawing it "unrolled" makes it look like a chain of copies, but it is one network applied again and again.
5Attention
A recurrent network squeezes everything it has read into one small vector h, and by the end of a long sentence the early words have faded. Large language models like GPT drop the recurrence. Instead, every word is allowed to look directly at every other word and decide which ones matter to it. That mechanism is attention.
Each word's vector x is turned into three smaller vectors by three weight matrices:
q = xWQ, k = xWK, v = xWV
The query q says what a word is looking for. The key k says what a word offers. The value v is what it passes on if chosen. A word scores every other word by the dot product of its query with their keys, turns the scores into percentages with softmax, and takes that blend of their values as its new vector:
Attention(Q, K, V) = softmax( QK⊤ / √d ) V
Dividing by √d, the square root of the key length, keeps the scores from getting so large that softmax gives one word everything.
In a real model the embedding has thousands of numbers instead of six labelled ones, and all the weights are learned. Each layer runs dozens of heads side by side, then sends every word's result through a dense network like the one in chapter 3. Stack a few dozen of those layers and read off the last word's vector, and you get a prediction for the next word. Generate that word, append it, and repeat: that loop is a large language model.