When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging

Researchers have introduced a new spectral analysis method to address task interference in continual learning and model merging. By examining the underlying geometry of weight updates, this approach aims to reduce catastrophic forgetting and improve performance when a single model is tasked with mastering multiple datasets sequentially.