Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters

Researchers have introduced Spec-AUF, a new training method for speculative decoding that addresses the performance gap between token drafters and target models. By using an accept-until-fail approach, this technique aligns the training of drafters with the specific way target models verify output, potentially increasing the efficiency of autoregressive text generation.