Language Models
Attention
Weighting what matters in context
About
A mechanism for relating tokens to each other. Query, key, and value projections, and how attention became the core of transformers.
Language Models
Weighting what matters in context
A mechanism for relating tokens to each other. Query, key, and value projections, and how attention became the core of transformers.