Change the stride, the filter, and the padding, and watch every number move. This page walks through a CNN one layer at a time: convolution, ReLU, pooling, and the final flatten, dense and softmax layers that turn feature maps into a prediction.
Each output value looks at a small patch of the input (the receptive field), not the whole image.
One small filter slides over the whole image, so the same pattern is detected anywhere with only K×K×C weights.
Early layers find edges, later layers combine them into textures, parts and whole objects. Pooling shrinks the maps as depth grows.
A convolution slides a kernel (filter) across the input. At each stop it multiplies the overlapping values element by element and adds them up. That single sum becomes one cell of the output, the feature map.
Hover over an output cell to move the kernel there. Hatched input cells are never visited by the kernel, which happens when (N + padding − K) is not divisible by the stride.
The numbers in a kernel decide what it responds to. A vertical-edge kernel gives large positive values where brightness rises from left to right and large negative values where it falls. A blur kernel averages neighbours, so it removes detail.
In a trained CNN nobody writes these numbers by hand. The network learns them with backpropagation, and a layer has many filters: Tiny VGG uses 10 per layer, VGG-16 uses up to 512. Each filter produces its own feature map, and the maps are stacked into the output channels.
ReLU is applied right after the convolution: ReLU(x) = max(0, x). It keeps positive responses and sets negatives to zero. Without a non-linearity, stacked convolutions would collapse into one linear operation.
Edit any kernel value, try the presets, or load your own photo. Orange is positive, blue is negative.
A pooling layer slides a window over each feature map, just like convolution, but it has no weights to learn. It summarises each window with one number.
Pooling works on each channel separately, so the number of channels stays the same. A 2×2 window with stride 2 halves the width and height and removes 75% of the values, which reduces computation and makes the network tolerant to small shifts.
After the convolution and pooling blocks, the network holds a small stack of feature maps. The last layers turn that stack into one probability per class.
Flatten unrolls the stack (H × W × C) into one long vector. It has no weights. Tiny VGG flattens 13 × 13 × 10 into 1,690 numbers.
Dense (fully connected) computes z = W·x + b. Every input connects to every output, so this is often where most of the parameters live: 1,690 × 10 + 10 = 16,910 in Tiny VGG.
Softmax turns the scores z (logits) into probabilities that add up to 1: pi = ezi / Σ ezj. The predicted class is the largest p. During training, cross-entropy loss compares p with the correct label.
Draw on the 12×12 input (click and drag). The network is hand-built so you can read every weight: 4 filters (3×3, same padding) → ReLU → max pool 2×2 → flatten 144 → dense 4 → softmax.
Positive weights reward activity in the class's own map, negative weights penalise the others. A trained network learns weights that vary across positions, but the idea is the same: each output neuron is a template over the flattened features.
Think of each output neuron as a judge listening to every input.
Method to understand it: pick one neuron, change one weight, and watch its score move. Then look at the matrix view. Each row of W is one neuron, so a whole dense layer is just many of these weighted votes computed at once.
Click an output neuron in the diagram to select it. Orange lines are positive weights, blue lines negative; thicker means larger.
A parameter is a number the network adjusts during training: every weight and every bias. Counting them tells you the model's size and memory cost.
One filter sees a K × K patch through all Cin input channels. The number of filters is the number of neurons.
One neuron sees every input. There is one neuron per output.
They apply a fixed rule (max, average, reshape, exponentials). Nothing is learned.
| Layer | Calculation | Parameters |
|---|---|---|
| Tiny VGG conv_1_1 | (3×3×3 + 1) × 10 | 280 |
| Tiny VGG conv_1_2 | (3×3×10 + 1) × 10 | 910 |
| Tiny VGG dense | (13×13×10 + 1) × 10 | 16,910 |
| VGG-16 first dense | (7×7×512 + 1) × 4096 | 102,764,544 |
Change the hyperparameters of Tiny VGG and see how every layer's output shape and parameter count responds.
Notice how "same" padding keeps maps larger, so the flatten vector and the dense layer grow. That is one reason modern networks replace flatten with global average pooling.
| Layer | Type | Output shape | Parameters |
|---|---|---|---|
| Total trainable parameters | |||
28×28. (32 − 5) / 1 + 1 = 28.
Same padding, P = 1 on every side. In general P = (K − 1) / 2 for odd K.
6×6 with valid padding (the last row and column are dropped), 7×7 with same padding.
448. (3 × 3 × 3 + 1) × 16. Each filter spans all input channels and has one bias.
Zero. It applies a fixed max or average and only reduces size.
The first class (argmax), and softmax outputs always sum to 1.