Transformer
In plain terms
A neural network architecture that powers modern language models.
Going deeper
Transformers use 'attention' to weigh how much each part of the input relates to every other part, processing sequences in parallel. They underpin models like GPT and BERT and revolutionized AI.
Under the hood
Self-attention scales quadratically with sequence length, spurring efficient variants (FlashAttention, sparse attention); positional encodings inject order since it's permutation-invariant.
Examples
GPT, BERT, T5, Vision Transformers, the 'Attention Is All You Need' architecture.
Think of it like...
Like a reader who, for every word, instantly glances back at all other words to grasp the full context before deciding what it means.