Attention Mechanism
In plain terms
Letting a model focus on the most relevant parts of input.
Going deeper
Attention weighs how much each input element matters to each output, enabling models to focus dynamically rather than treating everything equally. It captures relevant relationships.
Under the hood
Self-attention relates all positions to each other; the core innovation behind transformers.
Examples
Transformers, machine translation, LLMs, self-attention, focus weighting.
Think of it like...
Like a reader emphasizing the key words that matter most for understanding a sentence.