In the transformer architecture, what makes the decoder different from the encoder?

The Transformer decoder is distinct from the encoder due to its role in generating output sequences autoregressively. While both use self-attention, the decoder incorporates two key modifications:

1. Masked Self-Attention: The decoder's self-attention is masked to prevent attending to future tokens in the output sequence, ensuring that predictions are based only on previously generated tokens.
2. Encoder-Decoder Attention: The decoder includes an additional attention layer that allows it to attend to the output of the encoder. This enables the decoder to focus on relevant parts of the input sequence when generating each output token.