01 · Understand the idea
In simple words
A transformer is a model architecture that uses attention along with other components. Attention helps combine information from different positions in a sequence. In “river bank” and “bank account”, the surrounding words give different clues. This is an intuition for context, not a complete explanation of how a trained model works.
Two useful terms
Architecture: the organisation of a model. Attention: learned computation that weights information from different positions.
Build your understanding
Attention lets a model combine information from different positions using learned relevance weights. Transformers use attention with other components to process sequences. The representation of a word can change with context. This tutorial is an intuition exercise, not a full implementation of training or inference.
Sign in to save your reading progress.
Sign in