VisualizationsDeep Learning
Self-Attention
Hover a token to see where it attends. Adjust the temperature to sharpen or soften the attention distribution.
Hover a row to see where that token attends.
Each row is a query token; each cell is a softmax attention weight over the keys (rows sum to 1). Low temperature sharpens attention onto one token; high temperature spreads it out. Weights here come from random embeddings — it's the mechanism, not linguistics.