Relevant and Irrelevant: A Renormalization Group Analysis of Transformer Attention

Researchers have applied Wilsonian renormalization group theory to analyze how transformer attention mechanisms interact with trained multilayer perceptron residual stacks. The study aims to mathematically classify attention as a relevant, marginal, or irrelevant operator in the context of neural network stability and training dynamics.