EPnG: Adaptive Expert Prune-and-Grow for Parameter-Efficient MoE Fine-tuning

Researchers have introduced EPnG, a new fine-tuning method for Mixture-of-Experts models that dynamically prunes and grows experts during training. This approach addresses the inefficiency of traditional techniques by adjusting parameter allocation based on routing behavior rather than applying uniform updates across the entire model.