Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation

Sakana AI has developed a training method called Error Diffusion that creates neural networks without using backpropagation. By utilizing modulo error routing, these dual-stream networks operate according to Dale’s principle, which limits neurons to either excitatory or inhibitory signals. This approach demonstrates that machines can learn using biological principles, achieving 96.7% accuracy on MNIST and 61.7% on CIFAR-10. This research provides a potential alternative to standard weight transport methods that are not currently feasible in biological neural systems.