AI Glossary
Transformer
Architecture for language models that uses attention.
Definition
The transformer is the architecture that made modern language models possible. Its core is the attention mechanism: for each word, the model evaluates which other words in the sentence are important for it and weights them accordingly. This allows it to capture context and relationships across long texts. GPT, Claude, and Llama are all transformer-based models.
Related Terms
Sources: a16z AI Glossary · CNET AI Terms Glossary