How LLMs Are Trained
From raw text to a helpful assistant — the complete, beginner-first path behind models like Claude, Gemini, and GPT: tokens, transformers, pretraining, and RLHF.
—
The Big Picture
How LLMs Are Trained: The Big Picture
2hbeginner
Text into Numbers
Tokenization — Text into Tokens
3hbeginner
Text into Numbers
Embeddings — Tokens as Vectors
3hbeginner
The Transformer
Next-Token Prediction
3h 20mbeginner
The Transformer
Self-Attention from Scratch
5hintermediate
The Transformer
The Transformer Block
4h 20mintermediate
The Transformer
Build a GPT (nanoGPT)
6hintermediate
Pretraining at Scale
Pretraining Data & Curation
3h 40mintermediate
Pretraining at Scale
Scaling Laws & Compute
3h 40mintermediate
Pretraining at Scale
Pretraining the Base Model
5h 20madvanced
Post-training: Making it Helpful
Instruction Tuning (SFT)
4hintermediate
Post-training: Making it Helpful
Reward Models & Human Preferences
4h 20madvanced
Post-training: Making it Helpful
RLHF & DPO
5h 20madvanced
Post-training: Making it Helpful
Alignment & Constitutional AI
4h 20madvanced
Shipping the Model
Evaluating LLMs
3h 20mintermediate
Shipping the Model
Inference, Quantization & Serving
4h 20madvanced
Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.
Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.