About This Architecture
Feature Attention Module with Residual Gate combines dual-branch attention and feature enhancement to selectively amplify important input features. The Importance Branch generates attention scores via MLP and Softmax, while the Enhancement Branch produces refined features through learned transformations, with both streams merged via element-wise multiplication. A Sigmoid Gate controls the residual connection, enabling the network to learn when to apply attention-gated features or bypass them entirely. This architecture solves the problem of adaptive feature recalibration while maintaining gradient flow through residual connections. Fork this diagram on Diagrams.so to customize layer dimensions, activation functions, or integrate it into your transformer or CNN backbone.