The transformer is the architecture that made modern language models possible. Its core is the attention mechanism: for each word, the model evaluates which other words in the sentence are important for it and weights them accordingly. This allows it to capture context and relationships across long texts. GPT, Claude, and Llama are all transformer-based models.
In short
What is a transformer in AI?
The transformer is the architecture of modern language models. Its core is the attention mechanism: for each word, the model evaluates which other words are important and weights them. This allows it to capture context across long texts. GPT, Claude, and Llama are based on this architecture.
Using AI in your organisation?
We show you which applications are worth it for your organisation.