Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks

Researchers at Sakana AI have developed a training method called PC-ALM that allows neural networks to update layers locally rather than relying on backpropagation. By using a Lagrange multiplier to manage layer constraints, this technique enables the training of models with up to 1,000 layers without the computational bottlenecks typically associated with standard error propagation.