About This Architecture
Transformer encoder-decoder architecture with stacked multi-head self-attention, cross-attention, and feed-forward layers for sequence-to-sequence tasks. Source tokens flow through input embedding and positional encoding into an N-layer encoder stack using masked self-attention and residual connections, while target tokens follow a parallel path through the decoder with cross-attention to encoder outputs. The decoder stack projects attention outputs through a linear layer and softmax to generate token probabilities, enabling machine translation, summarization, and other conditional generation tasks. Fork this diagram on Diagrams.so to customize layer counts, embedding dimensions, or attention head configurations for your OCI-hosted model training pipeline. This architecture demonstrates the complete transformer pattern with residual connections and layer normalization essential for stable training of large language models.