MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for Transformers

Researchers from Apple have introduced MemoryLLM, an architecture designed to replace standard feed-forward networks in transformer models with an interpretable memory component. This approach aims to make the internal decision-making processes of large language models more transparent by decoupling memory storage from computational layers.