① Whole model
top to bottom = one forward pass
② One block, zoomed
repeated 32× with different weights
③ Inside the two sub-layers
attention and MLP
learned weights fixed before training plain math, nothing learned ⟲ dashed side line = residual (add input back)
Hover or tap any box for what it does, its shape, and whether it's learned. The sizes are from a real 8-billion-parameter open model (Llama 3 8B), which is typical of modern LLMs.

Where the 8 billion parameters actually live

attention
MLP (feed-forward)
0.53B · 6.5% embeddings
1.34B · 16.7% attention (Q, K, V, O)
5.64B · 70.2% MLPs
0.53B · 6.5% output head

Q/K/V is the idea that makes it a transformer, but most of the weights are in the MLPs. Layer-norm weights are tiny (about 0.3M in total).