Embarrassingly Simple Self-Distillation Improves Code Generation

Apple researchers have demonstrated that large language models can improve their own code generation performance by training on their own raw outputs. This method, known as simple self-distillation, functions without the need for external verification, teacher models, or reinforcement learning. By refining models through these filtered self-generated samples, developers may be able to increase coding accuracy using existing resources rather than requiring more complex training architectures.